Dreaming Of Beetles

Update: Patch 513747 has been applied.

The real hero of this story is Josh Aas. He got the 64-bit goodness into the codebase a couple weeks ago and committed it about a week later. I haven’t done any benchmarks on this so please check out Josh’s site – Boom Swagger Boom – and more specificly the benchmarks he’s put together. This isn’t a huge jump in performance, but I wasn’t really expecting one anyway. Since this version is using Gecko 1.9.3, there should already be a performance gain and the 64-bit goodness just adds to that.

The other interesting thing I’ve stumbled into is how to incorporate GCD into Firefox. I’m ready to start on that project, but have been bogged down with real world things like work. I’ll try to slip it in over this weekend or next, but no promises. There is an easy route and a hard route for GCD, I’m going to stick to the easy for now. As for OpenCL, that will be hard, very hard. I’ll look into that once my GCD builds are complete.

To download 64-bit Minfield, go to the Downloads page.

Popularity: unranked [?]

As you’ve probably noticed, I like to compile things. Package management, although useful, just really doesn’t do it for me. Recently, I had to set up a customized install of PHP on CentOS and was forced to use RPM. Well, the experience was funky to say the least. The basic php install had all compile-ins disabled. I had to root around and find the specific package, such as GD, then RPM it into PHP via an ini directive in /etc/php.d/. This sort of drove me nuts.

Anyway, onto Django. Snow Leopard comes with Python 2.6.1 and Apache 2.2.11. Of course I compiled my own version of Apache, resulting in a 2.2.13 version in /usr/local. I left Python alone. Installing Django was a snap and once again I had to go the bleeding edge route and use the Django trunk as my install, symlinking it into /Library/Python/2.6/site-packages/. This worked just great. Now I needed Apache to serve up my Django framework. I had the choice of mod_python or mod_wsgi and seeing the benefits of mod_wsgi, I went with that (also I’ve had horrible experiences with mod_python in the past).

I downloaded mod_wsgi, and did the following:

# tar -xzvf mod_wsgi-2.6.tar.gz
# cd mod_wsgi-2.6
# ./configure --with-apxs=/usr/local/apache2/bin/apxs
# make
# make install

And it blew up with the following error:

Warning! dlname not found in /usr/local/apache2/modules/ mod_wsgi.la.
Assuming installing a .so rather than a libtool archive.
chmod 755 /usr/local/apache2/modules/mod_wsgi.so
chmod: cannot access `/usr/local/apache2/modules/mod_wsgi.so’: No such file or directory
apxs:Error: Command failed with rc=65536
.
make: *** [install] Error 1

Great, mod_python nightmares started returning. I did some research and found many people were having problems with mod_wsgi on Snow Leopard. Graham Dumpleton, the maintainer has been very active in helping people out. His main suggesting was to modify the Makefile to have it point to the correct version of Python by modifying the LDFLAGS and LDLIBS. None of these suggestions worked for me, and even Graham seemed baffled as to why it would work on some Snow Leopard systems and not on others. One of his points was that people with a MacPorts build of Python were corrupting the mod_wsgi process as this build is a bit shaky. Without specifying your version of Python during ./configure, the MacPorts libraries would supersede the default install or other compiled versions. I checked an indeed I had a MacPorts Python in /opt/local/. How did that get there? I hate package managers.

But this wasn’t the problem as I could see the correct version of Python in the Makefile. Most of the people having problems had a customized Python build, but I was on the other side with a customized Apache build. So I looked into the difference between my APXS and the default APXS. The only major difference was on line 199:

($httpd = $0) =~ s:support/apxs$:httpd:;

My version did not have that httpd in there and looked like:

($httpd = $0) =~ s:support/apxs$::;

Making this change solved the problem. I’m not sure what I did to get a funky APXS, so if anyone knows I’d love to hear it.

I had some other issues with permissions, paths, and eggs, but they were fairly simple and if you follow the documentation, you can get everything working. I don’t always RTFM.

Hopefully this saves someone else the time and misery of setting up Django using mod_wsgi under Snow Leopard.

Popularity: unranked [?]

“Last night I received a Growl notification from Tweetie of an RT on a spam message I sent.”

My wife said “I don’t think I understood a word you said.” Well, that’s what happened and it didn’t dawn on me at first. With the number of followers I have, I get a constant stream of mentions, mostly spam. It finally sunk in that this person was attributing the message to me. First off, I don’t understand why someone would RT such a message, but I’m glad they did. Second, who, wha, huh? Why am I spamming people.

Try logging into my account. Fail. Try retrieving my password. Fail.

Sent a message to Twitter support around 1 AM, then started looking for ways to hack my account back. Using a separate account with API access, I started running through possible traps, DOS attacks, etc. About this time, I got the auto response from Twitter asking if I’m sure I didn’t just forget my password, so I respond to that and check my account again. It’s gone. Good. Better dead than spamming.

At about 6 AM, my account was reverted to my email address and the password was changed (I received no notification of this). I got in, secured the account and all is good again.

Why am I blathering about all this? OAuth.

Twitter has preventative measures against brute force attacks. If you fail logging in a certain amount of times, your account is locked for one hour. Not sure if this is IP based (which would be stupid) or just a general lockdown. There is no way somebody could have brute forced my password.

So how did it happen? Well, I signed up for a twitter stats service from a company I’ve heard positive things of in the past. They weren’t using OAuth so I exposed my credentials. In the early days of the Twitter ecosystem, this would be perfectly acceptable, but now after OAuth has been fully implemented and the exploits/bugs worked out, there is no excuse not to use it. If the developer is too lazy to implement OAuth, the service probably isn’t worth it. The company can be totally reputable (which is why I will not mention them), but if the passwords are stored, someone is gonna get tempted.

My personal feeling is that this ecosystem be FORCED to use OAuth. I even mentioned something of that nature on the dev-list. But the ecosystem is what matters and if a majority aren’t using OAuth or won’t upgrade, they’ll get what they want. Developers, Developers, Developers!

Personally, I’ll no longer use any service that isn’t using OAuth, and you should probably consider doing the same. Of course, there are other possible ways people could have compromised my account, but the timing just says otherwise.

Oh yeah, follow me on twitter!

image / Andrei

Popularity: unranked [?]

Update: And we have an answer – Google Maps Ditches Tele Atlas in Favor of Street View Cars and Crowdsourcing

I have no idea. But the title is catchy, and I’m hoping to learn the answer by a) writing out my thoughts and b) you. The transaction went down in July but was only uncovered recently, causing a day long brouhaha on the blog circuit, and now it’s been forgotten. As someone with a little knowledge in this area, I’m a tad more than intrigued. I’m just going to dive into a couple theories. They all focus on why Apple would move away from Google (I’m not even sure that is their intent):

1. Current Data Limitations
   Google places restrictions on their data use. A clear example of this is Jobs’ “BYOM”, Bring Your Own Map, statement about turn-by-turn directions. Google will not allow their data to be used for such functionality.
2. Cost Reduction
   Tiles aren’t free. Even to important and huge clients. Google went from a “per transaction” payment model to a “per tile” payment model and the entreprise level pricing isn’t exactly cheap. With the amount of Apple’s usage and even at a substantial discount, this cost is still probably in the tens to hundreds of millions per year.
3. More Control
   Apple is a control freak. No need to argue this. Being reliant on an increasingly competing company can’t feel good for anyone, especially Apple. The threat of Google barring Apple from their maps or even significantly altering the usage deal has DOJ written all over. This isn’t going to happen and Apple isn’t worried. Google does have free reign to make their maps look like crap if it affects all customers which leads into the next point…
4. Google Adding Data
   Just recently Google started adding advertisements PLUS user-generated content into iPhone apps. Wait, I thought one of the benefits of the enterprise license was to have the ability to remove ads. And wait, does this only affect iPhone apps? I think Google will come to their senses and flip off that UGC, unless you want to see it. The ad part is a bit odd and doesn’t jive with point 2 and 3 above.
5. More Control, Redux
   Google doesn’t own the tiles, they are licensed. Tele Atlas provides the map information, and other services provide the satellite imagery (Digital Globe, GeoEye, USGS, TerraMetrics, and the list goes on). There are some interesting things to note here.
   Tele Atlas provides data to many vendors including Yahoo and Microsoft and probably Placebase. Tele Atlas provides the map information, not the tiles, so each vendor can make them look however they want through Tele Atlas’ proprietary API. The United States is a 5GB or so text file. Apple’s designers could make their maps look better than everything else on the market, without a doubt.
   The satellite data comes mainly from Digital Globe, but once you start zooming in, watch the copyright info on the lower right of the map – the data comes from many, many sources. Apple can easily strike deals with these same sources. Digital Globe is happy to point out their non-exclusivity with Google. But on the other hand Google seems to be in bed with GeoEye, launching a satellite together and all.
   Other data such as traffic and street view is proprietary to Google.
6. Data Layers
   Placebase offers many layers of statistical data for their paid API and have won awards for their PolicyMap website. But if you look close enough, this is just tract data freely available from the US Census Bureau. Gathering and overlaying this data is a trivial task. In one of my previous companies, we built a much, much richer data set covering many more areas than Placebase offers. This was a hurculean task, but as a small startup, we pulled it off. So I think access to the data layers is moot here as it adds little value.

So what’s this all about then? My guess is that is has to do with Augmented Reality/Extended Mapping capabilites. This could be especially useful on the mythical tablet and could be even more useful for the collection of future Census Data. Not only in the US, but the rest of the world. It’s a little late for the 2010 Census though, so this idea is a tad suspicious.

I welcome all opinions, corrections, and comments on this as I really want to get to the bottom of Why Apple Bought Placebase.

Popularity: unranked [?]

Update: An interesting thing I noticed about the suspension process is that Twitter literally unfollows ALL your friends. And when you are reinstated, they refollow ALL your friends. I would think there would be some kind of suspended flag somewhere to avoid taxing the server like this, but that isn’t how they roll. With having just followed 93,000 people in a single day, you can imagine how many DMs I got (19k and climbing).

For reasons unknown, Twitter has suspended my main account, @clatko. I have submitted support tickets but have the feeling those are going straight to /dev/null. I’m so lost without my account, I’ve gone and done the most faux paus thing possible, appealed to the Twitter developer list. I don’t expect a response on there until Monday as usually Twitter devs don’t work on weekends.

Come Monday, I’m going to raise the storm of storms to get my account reactivated. I have a pretty strong battle plan and may need to call on my facebook friends and friendfeed subscribers. I’m not going out without a fight.

Oh, look. My account is back. Somebody somewhere must like me.

Thank you!

image infekted.it

Popularity: unranked [?]

I wasn’t aware of Clang and the transition to Low Level Virtual Machine compiler infrastructure. I knew about LLVM, but never saw a roadmap of Apple’s transition until I read the always excellent Ars Technica OS X Review. I’m just now setting up my development environment and was very tempted to use Clang to compile everything. At the moment, I’m pressed for time so I don’t want anything to go wrong. The 64-bit Apache/PHP tutorial is working great so far under Snow Leopard (just find the more recent version of the projects and tools).

If you want to set up Clang as your default compiler, throw this in your /etc/bashrc:

export CC=clang
export CFLAGS=-Qunused-arguments
export CPPFLAGS=-Qunused-arguments
export PATH=/Developer/usr/bin:$PATH

Anyway, I will eventually get around to recompiling with Clang and will report on my results. The first project will be Shiretoko and/or Namoroka.

Thanks for the tip Ajay Kapal.

You should follow me on twitter here.

Popularity: unranked [?]

Though WebKit trounces the competition on the SunSpider JavaScript Benchmark, it is a different story in everyday use. There seems to be some kind of memory issues that can result in the spinning pizza/beachball of death. As you open more tabs, this becomes more common. Also, when leaving WebKit open for extended periods (and not downloading a new nightly), CPU usage can go through the roof.

Because of these issues and the fact that I’ve been working on a project that requires a massive number of open tabs, I’ve moved to Camino. If you don’t know about Camino, go check it out. It’s a lightweight, cocoa browser with the Gecko 1.9.x rendering engine. And it kicks Firefox’s ass! There are a lot of things missing, but what you gain is speed. (Also, there are a ton of little things like being able to tab through links on a page).

I’ve been using the 2.0 Beta 3 release for a while and decided to push things a bit. Although Camino is already pretty well optimized for the Mac, I applied my Firefox build knowledge to try to take it further. While the JavaScript benchmarks are almost identical between my build and the official build, the overall browser experience seems to be faster. I know this is very subjective and means nothing, but the perceived difference is real for me.

So, I’m releasing an Intel-Optimized build on my downloads page. In the next week, I’ll upload my .mozconfig for Camino to see if there are further tweaks the community can point me to. For now, I just want to get the binary out there and some feedback.

So, comments please…

Popularity: unranked [?]

I originally published these thoughts in Seth Godin’s Triiibes community. I’m fleshing it out a tad for this post.

It is a well-known fact that John Rubinstein headed up the development of the original iPod and later took charge of the spun-out iPod division. He later left Apple to retire on some beach in Mexico where he was eventually hunted down by Roger McNamee of Elevation Partners. Roger made him an offer he couldn’t refuse and now we see Rubinstein leading the formerly beleaguered Palm. Rubinstein did NOT jump straight to Palm from Apple as has been reported on many sites.

This background, and not to mention the legions of Apple developers that moved to Palm (including a friend of mine), puts no doubt in anyone’s mind that inside knowledge of how to interface with iTunes came to Palm. Even so, they could have white-roomed it and figured it out on their own. I mean, how hard could this be?

With the PR frenzy leading up to the Pre’s release, it was noted that the Pre would natively synch with iTunes (and iPhoto), allowing you to sync anything that wasn’t DRM’d. At first, my opinion was that any device should have unfettered access to the non-DRM part of an iTunes library. However, the more I learned of the situation, the more my views have changed.

Consider this:

• There is an API for iTunes (doubleTwist is using this successfully and many others)
• Palm is blurring the support lines (if there is a synch problem, who is the customer going to call)
• If Apple lets Palm do this, everybody else will assume it as a right

I’m glad to see Apple finally move on blocking Palm. They should have done it a lot earlier. Also, I do not see this as being anti-competitive on Apple’s part, they do not have a monopoly on music catalog software (Songbird, anyone?). I see it more of inline with a DMCA violation on Palm’s part (though I know this is not the case).

What’s your take?

Popularity: unranked [?]

Update: Robert X. Cringely’s Op-Ed is out. Not at all what I expected, but that’s what I like about RXC. After watching a couple hours of Google IO ‘09, I finally realized the deal with Chrome OS. Learn more about Native Client and it will all make sense – Google IS revving up for a fight for the Desktop OS and NaCl is how they’ll do it. And I wish the “Tech” blogs would understand the difference between Linux, GNU/Linux, and a Distro. It’s almost embarrassing to read. Disregard my blathering below.

I must have read over 50 articles on this whole Chrome OS thing and I’m still baffled. Here are some of the better articles I’ve read on the subject:

• Putting What Little We Actually Know About Chrome OS Into Context (Daring Fireball)
• Let’s all take a deep breath and get some perspective (Fake Steve Jobs)
• Android, Chrome OS Relationship Confusing Everyone, Including Google (Gizmodo)

To see a more complete list of stuff I’ve read, take a look at my Recently Read Articles. I’m still waiting on what Robert X. Cringely is going to say (he loves to take his conclusions to the logical extreme), but he’s holding out for an Op-Ed piece in the New York Times.

Some interesting theories:

• Chrome OS is a way of sticking it to the man. By giving netbook/smartbook OEM’s another Windows alternative to use as a bargaining chip to lower the Windows tax.
• Chrome OS is meant as a dual-boot solution. It will be installed alongside any other operating system to give you a way to boot and jump on Gmail within 10 seconds.
• Chrome OS is meant to be a distraction for whatever Microsoft is set to announce on Monday.

What I think is going on, and what is hinted at in the Gizmodo link above, is that Google is using Chrome OS as FUD against itself. Android is creeping into a market for which Google never intended, the netbook/smartbook realm, and in order to prevent Android from seriously forking, Google is saying “We will have an OS for that space, don’t go there!” I’m not going to blather on about this because, as I said, I’m confused.

One other thought, any netbook/smartbook that Chrome OS goes on will be FREE. Not just the operating system, but the hardware – subsidized by Google and a minimal contract with a carrier.

image: Google

Popularity: unranked [?]

After living in Japan for six years and doing web programming for most of that time, you would think I would have this down by now. I used many combos – from Lasso/FileMaker to PHP/MSSQL and even PHP/PostgreSQL – but never used PHP/MySQL for any CJVK work. So I did some Googling and found four pages that claimed to have the answer:

1. Use UTF-8 No BOM for each page. That is Byte-Order Mark, which does help in other languages like Cold Fusion, but not for me in PHP. NOPE!
2. Use a PHP header tag:

   header('Content-Type: text/html; charset=utf-8');

   and use a HTML meta tag:

   NOPE!

3. Use SET NAMES ‘utf8′; when instantiating your database object. NOPE!
4. Change the column to utf8_general_ci and the collation to utf8_general_ci. NOPE!

I saw that PhpMyAdmin was displaying the characters correctly, so how were they doing it? I did a deep dive into the code and wound up at the mysql dbi connector where the following statements were set for EVERY query:

mysqliObj->query("SET CHARACTER SET 'utf8'");
mysqliObj->query("SET collation_connection = 'utf8_general_ci'");

This, along with the column set to utf8_general_ci did the trick. The processing pages were set to Western (Mac OS Roman) which did not cause any problems inserting or displaying Japanese data.

This post is more for myself so I don’t ever forget how this is done. This can be black magic sometimes so I need some documentation.

Popularity: unranked [?]