11.8.09

Wordpress 2.8.5 dashboard not working in Firefox

I’ve been playing with an install of Wordpress 2.8.5.. & along with a plethora of other bugs & issues, there was one in particular that stood out.

I couldn’t use the dashboard under Firefox.

It took a little searching, but I found the issue. Here’s how to solve it:

In your [blog root]/wp-admin/load-script.php file, around line 618, there is a line thus (this is all on one line, it’s just wrapping when displayed here):

echo “<script type=’text/javascript’ src=’” . esc_attr($src) . “‘></script>\n”;

change it to:

echo “<script type=’text/javascript’ src=’” . $src . “‘></script>\n”;

and around line 687 there is a line:

echo “<link rel=’stylesheet’ href=’” . esc_attr($href) . “‘ type=’text/css’ media=’all’ />\n”;

change it to:

echo “<link rel=’stylesheet’ href=’” . $href . “‘ type=’text/css’ media=’all’ />\n”;

So what’s going on here?

The lines before those two create urls for loading scripts & stylesheets, respectively.

Those urls are then getting run through esc_attr, which turns characters like & into strings like &amp;

Often this is useful, but not here. What it means is that the html that is getting expressed in the page (you can see this if you pull up the dashboard & do a view source) looks like this:

<script type=’text/javascript’ src=’http://[site]/wp-admin/load-scripts.php?c=1&amp;load=jquery,utils,quicktags&amp;ver=b64ae9a301a545332f1fcd4c6c5351b4′></script>

and

<link rel=’stylesheet’ href=’http://[site]/wp-admin/load-styles.php?c=1&amp;dir=ltr&amp;load=dashboard,plugin-install,global,wp-admin &amp;ver=6403d4cb3e6353f406fd43f1b0373ec2′ type=’text/css’ media=’all’ />

Which basically meant that the files weren’t getting loaded at all, the dashboard was looking like complete crap, and, well, not working at all.

The slight changes above simply remove that encoding, resulting in the correct urls:

<script type=’text/javascript’ src=’http://[site]/wp-admin/load-scripts.php?c=1&load=jquery,utils,quicktags&ver=b64ae9a301a545332f1fcd4c6c5351b4′></script>

and

<link rel=’stylesheet’ href=’http://[site]/wp-admin/load-styles.php?c=1&dir=ltr&load=dashboard,plugin-install,global,wp-admin &ver=6403d4cb3e6353f406fd43f1b0373ec2′ type=’text/css’ media=’all’ />

11.8.09

Firefox 3.5.5 screwy characters appearing

There’s something that’s bugged me ever since I upgraded to Firefox 3. Certain pages that used to work perfectly in Firefox 2 suddenly didn’t.

Instead there would be a mess on the page – lots of square boxes the size of characters with text inside them. Like this comp_1.jpg or maybe this comp_2.jpg

Typically this would be some kind of character encoding issue ( the server/browser specifying/requesting UTF-8 instead of ISO-8859-1 etc), or having Auto-Detect universal set off in Firefox – and most sites around the net propose this as a solution (oh, & also recommend partial reinstalls of your O/S).

Uhh, no.

It’s actually a compression issue.

If you’re having this problem, the resolution is this:

Enter into the address bar

about:config

in the Filter textbox below, type

network.http.accept-encoding

You can also just start typing “accept-encoding” until it appears on the screen.

Double click the network.http.accept-encoding entry.

Now, on my browser, it was set to

gzip,deflate;q=0.9,compress;q=0.7

but should have been

gzip,deflate

So, type that into the box & hit OK, then restart your browser (just make sure you close all your windows too)

Voila, you can now surf the web without having to constantly switch back to IE.

10.23.09

Twitter OAuth Invalid Signature on friendships/create

This is a public service announcement.

I’ve been doing a bunch of work with Twitter recently & came across this problem.

When trying to do a friendships/create, I get back “OAuth Invalid Signature.”

I’m using Tweetsharp v0.15 preview (an excellent product, btw), but I don’t think this is a Tweetsharp issue, it’s a Twitter issue. People are really scratching their heads about it.

The Tweetsharp guys proposed a solution here, but that didn’t help me. In fact, the more I googled, the more erroneous solutions I found.

Here’s my setup. TwitCleaner (the app) has a consumer keys & secret. It would then get an access token/secret for the user, & use that token/secret to make the user follow @TheTwitCleaner. This is done so we can DM the user when their report is done. We encourage people to unfollow again (if they want to) once they get their report DM.

Anyway, pretty simple. We have valid OAuth token/secret from the user, so that’s not a problem.

We’re just trying to make the user follow @TheTwitCleaner, should be simple, right? No.

I wasted several hours on this. Among the solutions proposed (& wrong) were:

  • You can’t use a consumer key/secret to follow the user those keys are associated with (ie, TwitCleaner the app has key/secret, but it’s associated with @TheTwitCleaner the Twitter account)
  • The OAuth information is incorrect
  • The request had to be made over https, not http (not something I have control over with TweetSharp, as far as I can tell)
  • That because I was passing in Client information when making the request, that was gumming things up.

Well guess what? It was none of those.

Know what fixed it?

Passing in the username to follow in lower case.

I kid you not.

Now, @TheTwitCleaner is in Twitter with that combination of upper/lower case, so I was passing it exactly as stored. But no, apparently befriend (Twitter API friendships/create) needs lower case in order to work reliably.

So now you know. Hope that saves you some pain.

03.30.09

The Importance Of Pipes

There’s a very subtle, often overlooked thing in Unix, the pipeline, or | character (often just called a pipe).

This is perhaps the most important thing in the entire operating system, with the possible exception of the “everything is a file” concept.

In case you’re unfamiliar (or didn’t feel like reading the wiki page above), here’s the basic concept:

A pipe allows you to link the output of one program to the input of another.

eg foo | bar – this takes the output from foo, and feeds whatever-it-is into bar – rather than, say, having to point bar at a specific file to make it do anything useful.

Why are pipes so awesome?

Well, the following reasons:

  1. Each program only has to do one thing, & do it well
  2. As such, development of those programs can be split up – even to the point where a thousand people can independently write a thousand programs, & they’ll all still be useful
  3. Each of those programs is very simple, thus faster to develop, easier to debug, etc
  4. Extremely complex behaviour can be created by linking different programs together in different ways
  5. None of that higher level behaviour has to be pre-thought or designed for

So, Unix has ended up with a ton of small but powerful programs. For example:

  • ls – lists a directory
  • cat – displays stuff
  • sort – sorts stuff
  • grep – finds things in stuff
  • tr – translates stuff (eg, upper to lower case)
  • wc – counts words or lines
  • less – pauses stuff, allowing forwards & backwards scrolling

I’ve been deliberately vague with the descriptions. Why? Because ’stuff’ can mean a file – if we specify it, or, it can mean whatever we pass in by putting a pipe in front of it.

So here’s an example. The file we’ll use is /usr/share/dict/words – ie, the dictionary.

cat /usr/share/dict/words

displays the dictionary

cat /usr/share/dict/words | grep eft

displays dict, but only shows words with ‘eft’ in them

cat /usr/share/dict/words | grep eft | sort

displays ‘eft’ words, sorted

cat /usr/share/dict/words | grep eft | sort | less

displays sorted ‘eft’ words, but paused so we can see what the hell we’ve got before it scrolls madly off the screen

cat /usr/share/dict/words | grep eft | grep -ve ‘^[A-Z]‘ | sort | less

displays paused sorted ‘eft’ words, but removes any that start with capital letters (ie, all the Proper Nouns)

cat /usr/share/dict/words | grep eft | grep -ve ‘^[A-Z]‘ | wc -l

gives us the count of how many non proper-noun ‘eft’ words there are in the dictionary (in the huge british english dictionary? 149, since I know you’re curious)

So there’s an additional benefit which is probably obvious. Debugging a complex set of interactions with pipes is incredibly straightforward. You can simply build up what you think you need, experimenting a little at each stage, & viewing the output. When it looks like what you want, you just remove the output-to-screen, and voila!

For the end-users, this means that the operating cost of using the system in a complex manner is drastically reduced.

What would happen without pipes? You’d end up with monolithic programs for every imaginable combination of user need. Ie, an unmitigated disaster. You can see elements of this in, umm, ‘certain other’ operating systems. *cough*

Most importantly of all, there is a meta benefit. A combination of all of the above benefits.

Pipes enable incredibly complex higher level behaviours to emerge without being designed in. It’s a spontaneous emergent behaviour of the system. There’s no onus on the system development programmers to be demi-gods, all they need to do is tackle one simple problem at a time – display a file, sort a file, and so on. The system as a whole benefits exponentially from every small piece of added functionality, as pipes then enable them to be used in every possible permutation.

It’s as if an anthill full of differently talented ants was suddenly building space ships.

Perhaps a better bits-vs-atoms metaphor is of money. Specifically the exchange of goods (atoms) for money, allows the conversion of those atoms into other atoms, via money. In the same way, pipes allows different programs to seamlessly interact via streamed data, in infinitely variable ways.

You don’t need to know how to make a car, since you can do what you’re good at, get paid, & exchange that money for a car. Or a boat. Or a computer. Society as a whole is vastly better off as each person can specialize & everybody benefits. Think how basic our world would be if we only had things that everybody knew how to build or do. Same thing with computers & pipes.

What seems like an almost ridiculously simple concept, pipes, has allowed an unimaginably sophisticated system to emerge from simple, relatively easily built pieces.

It’s not quite the holy grail of systems design, but it’s bloody close.

12.16.08

A Nifty Non-Replacing Selection Algorithm

Algorithms are awesome fun, so I was super pleased when my little bro asked me to help him with a toy problem he had.

The description is this: It’s a secret santa chooser. A group of people, where each person has to be matched up with one other person, but not themselves.

He’s setup an array that has an id for each person.

His initial shot was something like this (pseudo, obviously):

foreach $array as $key => $subarr {
  do {
      // $count is set to count($array)
      $var = rand(0, $count)
  } while $var != $key and $var isn't already assigned
  $array[$key][$assign] = $var
}

Initially he was mostly concerned that rand would get called a lot of times (it’s inefficient in the language he’s using).

However, there’s a ton of neat (non-obvious) problems with this algorithm:

  1. By the time we’re trying to match the last person, we’ll be calling rand (on average) N-1 times
  2. As a result, it’s inefficient as hell ( O(3N+1)/2)? )
  3. There is a small chance that on the last call we’ll actually lock – since we won’t have a non-dupe to match with
  4. Not obvious above, but he also considered recreating the array on every iteration of the loop *wince*

Add to this some interesting aspects of the language – immutable arrays (ie, there’s no inbuilt linked lists, so you can’t del from the middle of an array/list) & it becomes an interesting problem.

The key trick was to have two arrays:

One, 2-dimensional array (first dim holding keys, second the matches)
and one 1-dimensional array (which will only hold keys, in order).

Let’s call the first one “$list” and the second “$valid”.

The trick is this – $valid holds a list of all remaining valid keys, in the first N positions of the array, where initially N = $valid length. Both $list & $valid are initially loaded with all keys, in order.

So, to pick a valid key, we just select $valid[rand(N)] and make sure it’s not equal to the key we’re assigning to.
Then, we do two things:

  1. Swap the item at position rand(N) (which we just selected) with the Nth item in the $valid array, &
  2. Decrement N ($key_to_process).

This has the neat effect of ensuring that the item we just selected is always at position N+1. So, next time we rand(N), since N is now one smaller, we can be sure it’s impossible to re-select the just selected item.

Put another way, by the time we finish, $valid will still hold all the keys, just in reverse order that we selected them.

It also means we don’t have to do any array creation. There’s still a 1/N chance that we’ll self-select of course, but there’s no simple way of avoiding that.

Note that below we don’t do the swap (since really, why bother with two extra lines of code?) we simply ensure that position rand(N) (ie, $key_no) now holds the key we didn’t select – ie, the one that is just off the top of the selectable area.

Oh, and in this rand implementation rand(0, N) includes both 0 AND N (most only go 0->N-1 inclusive).

$valid = array_keys($list);
$key_to_process = count($valid) - 1;
do {
  $key_no = rand(0, $key_to_process);
  if ($key_to_process != $valid[$key_no]) {
    $list[$key_to_process][2] = $valid[$key_no];
    $valid[$key_no] = $valid[$key_to_process];
    $key_to_process--;
  }
  # deal with the horrid edge case where the last
  # $list key is equal to the last available
  # $valid key
  if ($key_to_process == 0 and $valid[0] == 0) {
    $key_no = rand(1, count($list) - 1);
    $list[0][2] = $key_no;
    $list[$key_no][2] = 0;
    $key_to_process--;
  }
} while ($key_to_process >= 0);


Without the edge-case code, this results in a super fast, nice slick little 10 or so line algorithm (depending on how/if you count {}’s :)

Elegant, I dig it.

12.13.08

Adobe AIR Locks If Installer Service Not Running

This is kindofa ridiculous post, but I Googled & couldn’t find anything, so this is primarily to help other people that might have this problem.

Adobe AIR apps will lock up (ie freeze) if you try to click a url (http link) in them and the Windows Installer service isn’t running.

I tested this with both in Twhirl (v 0.8.6 & 0.8.7) & Tweetdeck (v0.20b), on Windows 2000 (Win2k) SP4.

Typically if you start the service (either through the gui interface, or with a NET START MSISERVER run at dos or from Start-Run) after you’ve clicked a link, it’ll unlock the app.. but not always.

I have no idea why AIR needs to have the installer service running in order for it to connect to your browser (frankly I find it both suspicious & more than a little lame), but there we have it.

Hopefully this helps someone. And yes, this was a deliberately keyword heavy post :)

| Posted in Web | Comments
11.20.08

Auto Responders Are Good, Yes?

I recently had a very simple task – I wanted to get one of my brokers to send my daily account mails to a different email account.

That should be straight forward enough, right? Click the subscription information link, enter a new address, and on with my day?

Wooah Nelly, slow down there.

Some immediate problems:

  1. The email is completely blank, with a large attached pdf. That’s all, which means:
  2. No href links of ANY kind in the email, oh and:
  3. The one url in the pdf doesn’t match the ‘from’ domain
  4. This is a long dormant account, at a company that has been bought out three times since I last used it – so I have no phone contact information for them either (yes, I have tried to get them to only send me the monthly update, but they insist on their 30 identical dailies – but that’s a post for another time)

So, try the obvious – hit reply & request a change.

After two weeks of trying that, time for a new tack. I go to the website listed – a completely different company (it looks like they’ve outsourced their email blasting, which might explain their lack of flexibility) – pick the most likely looking email address I can find, and write a polite email.

A week later, still no response. Still getting these more or less useless (but oversized) emails to the wrong account.

So, time for a more aggressive approach. I go to the whois registry & get tech & biz contacts. I go to all the related websites. I get EVERY public email address I can find, and mail them all.

There’s seventeen of them.

I am polite – I explain that I realise I’m probably (after all, ONE of them may be the right one so I can’t be 100% certain) emailing the wrong person, but if they could please forward it on.. yadda yadda yadda.

I may have also *cough*accidentally*cough* mentioned that the CAN-Spam Act 2003 (yes, it’s an American company) makes it a legal requirement that there be working unsubscribe links available.

The fact that I’m emailing vice presidents & CEO’s in five countries I’m less concerned with – if they don’t hear about it, how will this ridiculously trivial problem ever get corrected? I don’t play golf with any of these guys.

Frankly I’m amazed that any company has public (& clickable, ie mailto) email addresses on the web considering how bad spam is these days – let alone vaguely high level people, but hey.

I get 7 responses, one justifiably aggrieved at being distracted from their Very Important Job. Mostly helpful, one super helpful who finally gets the job done.

And three bounces.

The only thing worse than posting email addresses publically is posting dead email addresses. Way to misunderestimate the web, guys.

But then, I’m guessing if they really ‘got’ the internet, they’d also have an auto-responder by now.

| Posted in Web | Comments
11.12.08

Old School Cool

Holey Moley this is so many flavours of cool I just had to post it:

I mean really, what’s not to like about that?

| Posted in Fun | Comments
09.16.08

The Trouble With Ratios

Ratios are used all over the place. No huge surprise there – they are, after all, just one number divided by another.

The well known problem case is when the denominator (the bottom bit) is zero, or very near zero. However, there are other subtler issues to consider.

Here’s a chart that has a ratio as the X axis:

ratio_pre.gif

Don’t sweat the details, they’re not terribly important – just the rough distribution.

The X axis in this case is what’s called a Calmar – ie, the total dollar return of a system divided by it’s maximum drawdown. Or, in English – how much you make proportional to how big your pockets need to be. This gives a non-dollar based (ie, “pure”) number that can then be compared across markets, systems, products, whatever.

This graph is actually a bit trickier than that, since there’s actually 3 dimensions of data there – it’s just the third dimension isn’t plotted – but we’ll get back to that.

Where this gets ugly is when, in the case of the Calmar above, the drawdown drops to, or near to, zero. For example, if you have a system that only trades once – and it’s a winning trade – the calmar will be very, very large. Even if you chuck out systems that are obviously a bit nutty like that, you can still end up with situations where the ratio has just blown out of all proportion.

Which results in this:

ratio_post.gif

See how everything is in a vertical line on the left?

Well, it’s not. Those points are actually quite well spread out – it’s just that instead of the X axis going from 0->50 as in the first case, it now goes from 0->22 million – of which only a small number are greater than a hundred (you can see them spread out on the right, very close to the Y axis)

In this example, we can see the problem, so we’re aware of it. However, what if the ratio had been the unplotted third dimension? We might never have known.

Now, the way that I’m using these ratios internally, I’m protected from these sorts of blowouts – I simply compare sets of ratios. If one is bigger, it doesn’t matter if it’s bigger by 2 or by 2 billion.

However, there are many situations where you might want proportional representation. If one value is twice as big, say, it should occur twice as often. In this case, ratios that explode out by orders of magnitudes quickly swamp results, and drive the whole thing into the ground.

You swiftly end up with a monoculture. One result eats all the others, and instead of a room full of happy spiders doing their thing, you end up with one fat angry spider in the middle of the room. Umm, so to speak.

Ratios can be dangerous, kids. Watch out!

08.2.08

Cuil Really Isn’t (Yet)

There’s been a lot of grumping about Cuil lately, so I thought I’d add to it (hopefully in interesting new ways).

I’ll talk in context of a site that I have *cough* some familiarity with: galadarling.com (hint: I built & managed it for the last two years). This site gets over 100k uniques a week, has a google pagerank of 6, and a technorati rank in the mid 5000’s. I.e., it’s not Yahoo, but it is a significant, popular smaller site.

The obvious test – surely searching for gala darling would return the site? It’s not complicated. Her name is the url. But no:

cuil_safe.jpg (click for a clearer view)

Maybe it’s the safe search? No, flick that off, and exactly the same results:

cuil_safe_off.jpg (click for a clearer view)

Putting in “gala darling” in quotes (just like that) results in the exact same result set. Huh?

Even scarier, putting in galadarling (all one word) doesn’t even return the site. How is this possible?

Even worse than that – gala.vox.com is the second returned result. This is a single page that was setup once and then ignored. It’s not just not finding the correct result, it’s actively returning junk.

None of these sets include Gala’s livejournal, which is updated every couple of weeks, let alone the actual site that has her name on it.

With the exception of her twitter account, all the results on the front page are other sites, talking about her… and this is useful.. how?

I looked at the first 20 pages of results – couldn’t find either her livejournal or main site. vox.com somehow managed to get several hundred mentions. A single collegecandy page appeared at least 5 times.

Ok, it’s common knowledge that the Cuil results are crap. How about some other things.

  • When you first load the page, you have to actually click to get into the textbox to enter your search terms.
  • For some reason I’m asked to accept cookies both from cuil.com (fair enough), and cuilimg.com (what? why?)
  • When paging through the results, there’s no way to go back to the start, as once you get past page 10, the earlier pages scroll off to the left, so you have to go backwards in chunks of 4 or 5.
  • There’s no way to get more results on a screen. Even on my 1900 pixel wide screen & a tiny font, I still only get 10ish results per page. Google allows me 100 – why should I click-wait-click-wait just to use Cuil?

To their credit, Cuil’s bot is not hitting the site anywhere near as much as it used to, so that’s one good thing. Considering galadarling is updated typically once a day, plus maybe a hundred comments, the Cuil bot (twiceler) used to hit the site about a thousand times a day (resulting in it getting blocked by damn near everyone). For comparison, Google’s bots hit it 400 times a day (partly that will be because there is context-sensitive advertising on the site, so Google needs to scan for that). Now Twiceler is visiting about a hundred times a day – much more reasonable given the update frequency.

I’ve talked to the guys running Cuil (back when it still had two ‘l’s in its name). They’re obviously very smart cookies and they definitely care about what they do. If they can shake Google up – well, great – and I say that as a Google shareholder. They definitely have a lot of bugs to iron out though, and the reliability of those results needs to be right at the top of the list. Without trust, what do they have?