Dreaming Of Beetles

Update: And we have an answer – Google Maps Ditches Tele Atlas in Favor of Street View Cars and Crowdsourcing

I have no idea. But the title is catchy, and I’m hoping to learn the answer by a) writing out my thoughts and b) you. The transaction went down in July but was only uncovered recently, causing a day long brouhaha on the blog circuit, and now it’s been forgotten. As someone with a little knowledge in this area, I’m a tad more than intrigued. I’m just going to dive into a couple theories. They all focus on why Apple would move away from Google (I’m not even sure that is their intent):

1. Current Data Limitations
   Google places restrictions on their data use. A clear example of this is Jobs’ “BYOM”, Bring Your Own Map, statement about turn-by-turn directions. Google will not allow their data to be used for such functionality.
2. Cost Reduction
   Tiles aren’t free. Even to important and huge clients. Google went from a “per transaction” payment model to a “per tile” payment model and the entreprise level pricing isn’t exactly cheap. With the amount of Apple’s usage and even at a substantial discount, this cost is still probably in the tens to hundreds of millions per year.
3. More Control
   Apple is a control freak. No need to argue this. Being reliant on an increasingly competing company can’t feel good for anyone, especially Apple. The threat of Google barring Apple from their maps or even significantly altering the usage deal has DOJ written all over. This isn’t going to happen and Apple isn’t worried. Google does have free reign to make their maps look like crap if it affects all customers which leads into the next point…
4. Google Adding Data
   Just recently Google started adding advertisements PLUS user-generated content into iPhone apps. Wait, I thought one of the benefits of the enterprise license was to have the ability to remove ads. And wait, does this only affect iPhone apps? I think Google will come to their senses and flip off that UGC, unless you want to see it. The ad part is a bit odd and doesn’t jive with point 2 and 3 above.
5. More Control, Redux
   Google doesn’t own the tiles, they are licensed. Tele Atlas provides the map information, and other services provide the satellite imagery (Digital Globe, GeoEye, USGS, TerraMetrics, and the list goes on). There are some interesting things to note here.
   Tele Atlas provides data to many vendors including Yahoo and Microsoft and probably Placebase. Tele Atlas provides the map information, not the tiles, so each vendor can make them look however they want through Tele Atlas’ proprietary API. The United States is a 5GB or so text file. Apple’s designers could make their maps look better than everything else on the market, without a doubt.
   The satellite data comes mainly from Digital Globe, but once you start zooming in, watch the copyright info on the lower right of the map – the data comes from many, many sources. Apple can easily strike deals with these same sources. Digital Globe is happy to point out their non-exclusivity with Google. But on the other hand Google seems to be in bed with GeoEye, launching a satellite together and all.
   Other data such as traffic and street view is proprietary to Google.
6. Data Layers
   Placebase offers many layers of statistical data for their paid API and have won awards for their PolicyMap website. But if you look close enough, this is just tract data freely available from the US Census Bureau. Gathering and overlaying this data is a trivial task. In one of my previous companies, we built a much, much richer data set covering many more areas than Placebase offers. This was a hurculean task, but as a small startup, we pulled it off. So I think access to the data layers is moot here as it adds little value.

So what’s this all about then? My guess is that is has to do with Augmented Reality/Extended Mapping capabilites. This could be especially useful on the mythical tablet and could be even more useful for the collection of future Census Data. Not only in the US, but the rest of the world. It’s a little late for the 2010 Census though, so this idea is a tad suspicious.

I welcome all opinions, corrections, and comments on this as I really want to get to the bottom of Why Apple Bought Placebase.

Popularity: unranked [?]

Update: An interesting thing I noticed about the suspension process is that Twitter literally unfollows ALL your friends. And when you are reinstated, they refollow ALL your friends. I would think there would be some kind of suspended flag somewhere to avoid taxing the server like this, but that isn’t how they roll. With having just followed 93,000 people in a single day, you can imagine how many DMs I got (19k and climbing).

For reasons unknown, Twitter has suspended my main account, @clatko. I have submitted support tickets but have the feeling those are going straight to /dev/null. I’m so lost without my account, I’ve gone and done the most faux paus thing possible, appealed to the Twitter developer list. I don’t expect a response on there until Monday as usually Twitter devs don’t work on weekends.

Come Monday, I’m going to raise the storm of storms to get my account reactivated. I have a pretty strong battle plan and may need to call on my facebook friends and friendfeed subscribers. I’m not going out without a fight.

Oh, look. My account is back. Somebody somewhere must like me.

Thank you!

image infekted.it

Popularity: unranked [?]

I wasn’t aware of Clang and the transition to Low Level Virtual Machine compiler infrastructure. I knew about LLVM, but never saw a roadmap of Apple’s transition until I read the always excellent Ars Technica OS X Review. I’m just now setting up my development environment and was very tempted to use Clang to compile everything. At the moment, I’m pressed for time so I don’t want anything to go wrong. The 64-bit Apache/PHP tutorial is working great so far under Snow Leopard (just find the more recent version of the projects and tools).

If you want to set up Clang as your default compiler, throw this in your /etc/bashrc:

export CC=clang
export CFLAGS=-Qunused-arguments
export CPPFLAGS=-Qunused-arguments
export PATH=/Developer/usr/bin:$PATH

Anyway, I will eventually get around to recompiling with Clang and will report on my results. The first project will be Shiretoko and/or Namoroka.

Thanks for the tip Ajay Kapal.

You should follow me on twitter here.

Popularity: unranked [?]

Though WebKit trounces the competition on the SunSpider JavaScript Benchmark, it is a different story in everyday use. There seems to be some kind of memory issues that can result in the spinning pizza/beachball of death. As you open more tabs, this becomes more common. Also, when leaving WebKit open for extended periods (and not downloading a new nightly), CPU usage can go through the roof.

Because of these issues and the fact that I’ve been working on a project that requires a massive number of open tabs, I’ve moved to Camino. If you don’t know about Camino, go check it out. It’s a lightweight, cocoa browser with the Gecko 1.9.x rendering engine. And it kicks Firefox’s ass! There are a lot of things missing, but what you gain is speed. (Also, there are a ton of little things like being able to tab through links on a page).

I’ve been using the 2.0 Beta 3 release for a while and decided to push things a bit. Although Camino is already pretty well optimized for the Mac, I applied my Firefox build knowledge to try to take it further. While the JavaScript benchmarks are almost identical between my build and the official build, the overall browser experience seems to be faster. I know this is very subjective and means nothing, but the perceived difference is real for me.

So, I’m releasing an Intel-Optimized build on my downloads page. In the next week, I’ll upload my .mozconfig for Camino to see if there are further tweaks the community can point me to. For now, I just want to get the binary out there and some feedback.

So, comments please…

Popularity: unranked [?]

With all this babbling about Verizon rushing out LTE for some non-dongle device and talks of the AT&T/Apple love affair breaking down, a simple question keeps popping to mind:

What happens when you don’t get LTE coverage? Aren’t you just going to get thrown down to the old CDMA? And the iPhone/iTablet is going to magically work with CDMA? Will the next gen iPhone have BOTH GSM & CDMA? Why is this never considered in any of the articles I read?

Popularity: unranked [?]

I originally published these thoughts in Seth Godin’s Triiibes community. I’m fleshing it out a tad for this post.

It is a well-known fact that John Rubinstein headed up the development of the original iPod and later took charge of the spun-out iPod division. He later left Apple to retire on some beach in Mexico where he was eventually hunted down by Roger McNamee of Elevation Partners. Roger made him an offer he couldn’t refuse and now we see Rubinstein leading the formerly beleaguered Palm. Rubinstein did NOT jump straight to Palm from Apple as has been reported on many sites.

This background, and not to mention the legions of Apple developers that moved to Palm (including a friend of mine), puts no doubt in anyone’s mind that inside knowledge of how to interface with iTunes came to Palm. Even so, they could have white-roomed it and figured it out on their own. I mean, how hard could this be?

With the PR frenzy leading up to the Pre’s release, it was noted that the Pre would natively synch with iTunes (and iPhoto), allowing you to sync anything that wasn’t DRM’d. At first, my opinion was that any device should have unfettered access to the non-DRM part of an iTunes library. However, the more I learned of the situation, the more my views have changed.

Consider this:

• There is an API for iTunes (doubleTwist is using this successfully and many others)
• Palm is blurring the support lines (if there is a synch problem, who is the customer going to call)
• If Apple lets Palm do this, everybody else will assume it as a right

I’m glad to see Apple finally move on blocking Palm. They should have done it a lot earlier. Also, I do not see this as being anti-competitive on Apple’s part, they do not have a monopoly on music catalog software (Songbird, anyone?). I see it more of inline with a DMCA violation on Palm’s part (though I know this is not the case).

What’s your take?

Popularity: unranked [?]

Update: Robert X. Cringely’s Op-Ed is out. Not at all what I expected, but that’s what I like about RXC. After watching a couple hours of Google IO ‘09, I finally realized the deal with Chrome OS. Learn more about Native Client and it will all make sense – Google IS revving up for a fight for the Desktop OS and NaCl is how they’ll do it. And I wish the “Tech” blogs would understand the difference between Linux, GNU/Linux, and a Distro. It’s almost embarrassing to read. Disregard my blathering below.

I must have read over 50 articles on this whole Chrome OS thing and I’m still baffled. Here are some of the better articles I’ve read on the subject:

• Putting What Little We Actually Know About Chrome OS Into Context (Daring Fireball)
• Let’s all take a deep breath and get some perspective (Fake Steve Jobs)
• Android, Chrome OS Relationship Confusing Everyone, Including Google (Gizmodo)

To see a more complete list of stuff I’ve read, take a look at my Recently Read Articles. I’m still waiting on what Robert X. Cringely is going to say (he loves to take his conclusions to the logical extreme), but he’s holding out for an Op-Ed piece in the New York Times.

Some interesting theories:

• Chrome OS is a way of sticking it to the man. By giving netbook/smartbook OEM’s another Windows alternative to use as a bargaining chip to lower the Windows tax.
• Chrome OS is meant as a dual-boot solution. It will be installed alongside any other operating system to give you a way to boot and jump on Gmail within 10 seconds.
• Chrome OS is meant to be a distraction for whatever Microsoft is set to announce on Monday.

What I think is going on, and what is hinted at in the Gizmodo link above, is that Google is using Chrome OS as FUD against itself. Android is creeping into a market for which Google never intended, the netbook/smartbook realm, and in order to prevent Android from seriously forking, Google is saying “We will have an OS for that space, don’t go there!” I’m not going to blather on about this because, as I said, I’m confused.

One other thought, any netbook/smartbook that Chrome OS goes on will be FREE. Not just the operating system, but the hardware – subsidized by Google and a minimal contract with a carrier.

image: Google

Popularity: unranked [?]

After living in Japan for six years and doing web programming for most of that time, you would think I would have this down by now. I used many combos – from Lasso/FileMaker to PHP/MSSQL and even PHP/PostgreSQL – but never used PHP/MySQL for any CJVK work. So I did some Googling and found four pages that claimed to have the answer:

1. Use UTF-8 No BOM for each page. That is Byte-Order Mark, which does help in other languages like Cold Fusion, but not for me in PHP. NOPE!
2. Use a PHP header tag:

   header('Content-Type: text/html; charset=utf-8');

   and use a HTML meta tag:

   NOPE!

3. Use SET NAMES ‘utf8′; when instantiating your database object. NOPE!
4. Change the column to utf8_general_ci and the collation to utf8_general_ci. NOPE!

I saw that PhpMyAdmin was displaying the characters correctly, so how were they doing it? I did a deep dive into the code and wound up at the mysql dbi connector where the following statements were set for EVERY query:

mysqliObj->query("SET CHARACTER SET 'utf8'");
mysqliObj->query("SET collation_connection = 'utf8_general_ci'");

This, along with the column set to utf8_general_ci did the trick. The processing pages were set to Western (Mac OS Roman) which did not cause any problems inserting or displaying Japanese data.

This post is more for myself so I don’t ever forget how this is done. This can be black magic sometimes so I need some documentation.

Popularity: unranked [?]

Minefield

I’ve built a pre-Alpha 1 version of Firefox 3.6 also known as Namoroka (following the park theme). Namoroka is a national park in Madagascar.

As with previous Shiretoko builds I’ve done, I’m using the same .mozconfig file for optimization as well as fiddling with the application defaults for speed. I am no longer changing the User-Agent on these builds as some people are reporting problems when accessing sites such as Facebook that don’t recognize the UA. When are people going to stop browser sniffing?

In this version, the “parse HTML 5″ was off by default. I’ve turned this on. One other thing is that the browser’s name is “Minefield” as this is a pre-Alpha and things are supposed to blow up (so be careful).

You can find the Namoroka build on my downloads page.

Popularity: unranked [?]

Follow Like Crazy

Update: I have confirmed that Twitter is using MySQL (unless the mentioned upgrades are a move to a different DB).

With all due respect, Michael Jackson may have given Twitter the final knockout punch. I’m sure you guys are sick of hearing of Twitter’s problems, and frankly I’m sick of writing about them. This new failure is worse than any failwhale, this is a failure of concurrency. Here is a brief timeline of what has happened so far:

1. June 3 – Twitter realizes there is a half-hour lag on follow/unfollow that should be resolved WITHIN THE NEXT HOUR OR SO.
2. June 18 – Twitter states it is making infrastructure upgrades to fix the follow/unfollow delay OVER THE NEXT 24 HOURS.
3. June 23 – Twitter again states there is a lag on follow/unfollow and offers additional info on areas affected – device notification changes and favoriting, BUT WE ALREADY KNEW THIS.
4. June 23 – Twitter announces additional upgrades for the 24th to fix the problem and says the problem will persist UNTIL LATER IN THE DAY TOMORROW.
5. June 24 – Twitter says the upgrades were successful and that the catch up period will last FOR AT LEAST ANOTHER 6-12 HOURS.
6. June 25, 11:26 am – A Twitter employee states “We’re still working on the fix and this is currently the top priority of the services team. It’s a pretty extensive code deployment so it is taking some time.”
7. June 26 – I’m following fewer people today than I was yesterday and my 1,000 FOLLOWING/DAY LIMIT HAS BEEN HIT.

I can’t talk about the Twitter infrastructure, but I’ve seen this problem before in one of my own companies. With MySQL replication across multiple servers and tons of activity going on, it is almost impossible for the slaves to catch up. In addition, each replicator is generating enormous log files in the event replication fails. These log files can quickly fill up a server especially if you don’t know what you’re doing and have MySQL in a tiny /var partition. Once the log file overruns the server, you cease to replicate until the situation is rectified. I suspect this is what the problem is and with each addition of servers (in the above-mentioned upgrades), those log files get nastier and nastier. There is a fix for this replication problem, but it involves taking all systems offline, rsynching from a master (if there is one) and clearing all logs.

Now MJ steps into the picture and blows the infrastructure away. The search sidebar was removed and later re-added, but this just keeps the failwhale at bay and does nothing but compound the follow/unfollow delays. Now we’re at critical mass with this problem as follow/unfollow basically does not work, or works inconsistently at best. This is going to turn people off in droves as the system is not working as expected. The “I just don’t get it” of Twitter has just been amplified. The image on the above-right show a single person that REALLY wants to follow me, each mail highlighted in blue says “This person has just followed you,” sad thing is, after all this effort they still aren’t following me.

With the failwhale, people got upset but realized that there is so much cool stuff going on here I can hang tight until the system is back. It is sort of like the logic behind the beta-invite. This is entirely different, Twitter isn’t acting as expected.

Also, I’m only taking about one aspect of Twitter in this post. There are the search problems which still aren’t fixed despite the update provided in that link. There was the “all posts coming from the web” problem which occurred over a weekend where, apparently, they take a holiday. This may not sound like a big deal, but it was for a lot of developers and even one business had to SHUT DOWN until the problem was fixed. There are many, many other issues that I’m just not going to bring up.

I’m not giving up on Twitter, but you can find me on FriendFeed.

To Twitter’s credit they have been fairly open on their status blog and their employees are pretty active on the mailing lists.

Popularity: unranked [?]