For various reasons, I have removed nofollow from comments. Some of the reasons:
Popularity: unranked [?]
If you are fed up with Gmail and want to pull all your mail, here is how you do it. This technique was used on over 30 mail accounts so I’m sure it will work for you.
The problem of exporting your mail from Gmail is not a trivial one. From discussions by Opera Software’s lead QA for Opera Mail’s posting on Gmail’s Buggy IMAP Implementation to Matt Cutts’ posting on How to back up your Gmail on Linux in four easy steps to LifeHacker’s posting on Back up Gmail on Linux with Getmail to Wired’s wiki entry on Make a Local Backup Of Your Gmail Account, it seems that there is no single definitive source on how to pull your mail and retain your labels.
So here is what I’ve done to solve this problem:
data = self.sslobj.read(size-read)
with
data = self.sslobj.read(min(size-read, 16384))
to maintain a 15MB memory block if necessary.
[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
mailboxes = ("[Gmail]/Starred",)
username = username@yourdomain.com
password = xxx
[destination]
type = Maildir
path = ~/Maildir/
[options]
verbose = 2
message_log = ~/.getmail/gmail.log
First of all, we are using IMAP to retrieve mail as POP has a limit of 99 documents per access and that would take forever.
Second, we are using the Maildir format for the destination so we need to make sure the target directories have been created (~/Maildir/cur, ~/Maildir/new, ~/Maildir/tmp).
Third, we need to specify a mailbox or mailboxes to download or the INBOX will be the default.
Fourth, we need a trailing comma on the list of mailboxes to download due to a parsing error in getmail (actually the mailboxes option needs to be a tuple, but the trailing comma negates that).
Fifth, we need to know the syntax of Gmail’s internal IMAP structure to pull down discrete folders. Non-label folders (Starred, Sent Mail, Drafts, etc.) are accessed with “[Gmail]/Starred” (as in the above config) and labels are accessed directly. For example, the label “Important Project” would have this in the config:
mailboxes = ("Important Project",)
/* VARS ***********************************************************/
$box = '';
$stem = SITE_DIR.'Maildir/'.$box.'/new/';
/******************************************************************/
$dir_contents = scandir($stem);
foreach($dir_contents as $item) {
if(!ListFind('.,..,.DS_Store',$item)) {
$file = $stem.$item;
$content = file_get_contents($file);
$date = extractText($content,"nDate: ","n");
$utime = strtotime($date);
$converted = date('YmdHi.s',$utime);
shell_exec('touch -mt '.$converted.' "'.$file.'"');
}
}
function extractText($content,$start,$end) {
if(strripos($content,$start)===false) { return false; }
$startpoint = strripos($content,$start)+strlen($start);
$endpoint = strripos($content,$end,$startpoint);
$length = $endpoint - $startpoint;
return trim(substr($content,$startpoint,$length));
}
function ListDeleteAt($inList, $inPosition, $inDelim = ',') {
$aryList = _listFuncs_PrepListAsArray($inList, $inDelim);
array_splice($aryList, $inPosition-1, 1);
$outList = join($inDelim, $aryList);
return $outList;
}
function _listFuncs_PrepListAsArray($inList, $inDelim) {
$inList = trim($inList);
$inList = preg_replace('/^' . preg_quote($inDelim, '/') . '+/', '', $inList);
$inList = preg_replace('/' . preg_quote($inDelim, '/') . '+$/', '', $inList);
$outArray = preg_split('/' . preg_quote($inDelim, '/') . '+/', $inList);
if(sizeof($outArray) == 1 && $outArray[0] == '') {
$outArray = array();
}
return $outArray;
}
photo: chris ivarson
This is a reprint of a post I originally made at http://www.propertymaps.com/blog. I felt it was relevant to the current Gmail posts so am reprinting with slight modifications.
Popularity: unranked [?]
First off, I’m going to start posting in a new format – a “squirt” (screw you zune). I have thoughts to convey that are too long for a tweet and too short for a full entry. I’ll put all these squirts into a single category.
Here are some things about Safari that are chapping my hide:
Don’t get me wrong, there is plenty of nice eye candy, especially when you load up the browser for the first time. I’ve seen a lot of the “150 New Features” before as I’ve been compiling this version for a while now. Of those 150 features, there are probably 20 that are new and most I don’t really care about.
Anyway, WebKit has been and will continue to be my default browser (as I write this in Shiretoko).
image: apple
Popularity: unranked [?]
Gmail Account Lockdown
While everyone is up in arms about the recent Gmail outage, I was up in arms about my Account Lockdown (pictured above). Mail.app has been acting up on me again lately (I have a post about Mail.app), so I decided I would clear out my junk in Gmail. I had about 150,000 messages in there and wanted to get rid of about 90% of it so I started mass deleting via IMAP.
Before long, I started noticing that nothing was actually happening. Mail.app was moving the messages to Trash, but when I would pop back over to my “All Mail” folder, the same number of messages were there. WTF? So I decide to log in to the web interface to check things out (and to check out the new button styles) and lo and behold, my account was locked down.
The lockdown was over “suspicious” behavior and I guess in my case that behavior was “deleting large amounts of email”. But I was doing it via IMAP, not POP. So off to the troubleshooting page which was of no help whatsoever, just telling me the lockdown would be in effect for 24 hours. Great.
This, the recent outage, and disappearing email has me slightly worried.
Popularity: unranked [?]
I did a little experiment and created a tweet bot that would announce every article I just finished reading. There were two problems with this – 1) I was putting out two much noise in my twit stream, 2) I may not necessarily want everyone to know when I’m reading and not working, heh. So I canned the idea.
Poking around these APIs (Twitter and Bit.ly) really opened my eyes quite a bit as to how powerful these simple services can be once the internals are exposed. Bit.ly ran an API contest and some of the entrant applications will blow your mind. A simple URL shortener can produce so much power. I wish I would have know about the contest a little earlier.
What I’m going to show here is almost the same as the 140it.com application, but my version is much more simplified. I also want to test a syntax highlighting plugin for PHP and many other languages which uses GeSHi.
So, I’ll just jump to the code and not go into the application logic. Here is the first part to shorten a URL:
// get shortUrl
$ch = curl_init();
$url = 'http://api.bit.ly/shorten';
$post_string = 'longUrl='.urlencode($data['url']).'&version=2.0.1';
$options = array(
CURLOPT_URL => $url,
CURLOPT_USERPWD => '[username]:[password]',
CURLOPT_POSTFIELDS => $post_string,
CURLOPT_RETURNTRANSFER => true
);
curl_setopt_array($ch, $options);
$response = curl_exec($ch);
curl_close($ch);
$response = json_decode($response);
if($response->statusCode!='OK') {
return false;
}
$shortUrl = $response->results->$data['url']->shortUrl;
if(trim($shortUrl)=='') {
return false;
}
And the next part to tweet about it:
// tweet that
$ch = curl_init();
$url = 'http://twitter.com/statuses/update.json';
$post_string = 'status='.urlencode('Just Read: '.$data['name'].' - '.$shortUrl);
$options = array(
CURLOPT_URL => $url,
CURLOPT_USERPWD => '[username]:[password]',
CURLOPT_POSTFIELDS => $post_string,
CURLOPT_RETURNTRANSFER => true
);
curl_setopt_array($ch, $options);
$response = curl_exec($ch);
curl_close($ch);
$response = json_decode($response);
if(isset($response->error)) {
return false;
}
If you decide to use this code, make sure your tweet is below the character limit. The above code does not reflect that.
One last thing, the real-time link stats at Bit.ly are the bomb. These guys are really pushing the boundaries here and I’m a huge fan. To access the real time stats, take an example URL like http://bit.ly/mQcHJ and add “info/” in the middle – http://bit.ly/info/mQcHJ.
Another last thing, the GeSHi plugin is also pretty cool. When copying the code, you don’t end up copying the line numbers.
image: bit.ly
Popularity: unranked [?]
Update: If the Envelope index trick doesn’t work for you, disable any bundles you have by moving them to the “Bundles (Disabled)” folder within ~/Library/Mail/. GrowlMail and Safari 4 Public Beta do not get along so you will have to disable this one.
I’m running two plugins (bundles) for Mail.app – GrowlMail and WideMail – and have experience zero problems. Once in a while, I get Mail.app in a frenzied state where it crashes if I breathe on it wrong. The console usually gives me nothing interesting, except maybe a message such as:
Mail[10935]: deleting invalid message from outbox
So I’m stuck with a flaky app. The usual cause of this is an inconsistent state in the mail index resulting from various things like trying to delete tens of thousand of server error messages from Gmail IMAP, or force quitting during a delete, or some other dumb act on my part.
To fix this, move “Envelope Index” from ~/Library/Mail/ and launch Mail. You’ll get an import dialog, hit “Continue” and wait. I have 322,984 messages to import so it will take a little more than a half an hour. Once done, you’ll usually see the corrupted message somewhere, usually in your spam or trash. Delete it (this step isn’t actually necessary for the fix).
Once you verify things are working, go ahead and delete the old “Envelope Index”, the import process created a new one for you.
image: Apple
Popularity: unranked [?]
Chromium on OS X
Update: Chromium build 12558 is now available.
Since Mike Pinkerton published some screenshots of Chromium running on OS X, there seems to be a little uproar in the Mac community about when we will see the final version. I decided to investigate a tad and built my own version for OS X. My version, rev 9780, seems to do a bit better than Mike’s because clicking on links work. This seems far from complete though, so if you enjoy lots of unexpected quits, this app is for you.
There is another app that is built with Chromium, called TestShell. It is a version of Chrome with basically no interface – this is the version the Mac developers have been using to port the backend code over. TestShell is quite a bit more stable than Chromium, but you don’t get any of the features. What you are seeing on Pinkerton’s site are the first versions of the Mac look and feel being added.
Some interesting things I noticed in the Chromium source code (I didn’t spend too much time looking):
Other findings:
Ok, on to benchmarks:
The updated Chromium build whomps WebKit with a 691 millisecond score. The above benchmarks are for the older version of Chromium.
Because this project is being developed so rapidly, I will start doing nightlies. Give me a few days to set that up.
Popularity: unranked [?]
Installing Tomcat is relatively easy as most of the work has already been done for you. These instructions pertain to OS 10.5.6 and will probably work for OS 10.5.x and 10.4.x. Going before 10.4 will probably required you to install a newer JDK. I’m going to run through this as root again. Any time you see #, this is the command line prompt. Ok, let’s get started:
1) Check Apache
Find where httpd is with “find httpd”. You are either running the default in /usr/sbin/httpd or running your own version in /usr/local/apache2/ or similar. I’m going to assume you are running in /usr/local/apache2/ and that you followed my other tutorial on installing 64-bit Apache/PHP. Now check what modules you have installed with “/usr/local/apache2/bin/httpd -l” and you should see a list of mod_* entries.
We’re going to stick with the reverse proxy and not go the mod_jakarta route (trust me, this is easier and I’m not even sure mod_jakarta is even being worked on anymore). So what we need to see in this mod_* list is mod_rewrite and mod_proxy. If you followed my previous instructions, you’ll note that mod_proxy is NOT there. We will have to recompile Apache.
2) Compile Apache
Go into /usr/local/src/httpd-2.2.11/config.nice and add the line
"--enable-proxy"
right after the line
"--enable-rewrite"
then from the command line:
# ./config.nice
# make
# make install
This will only compile/install the bits needed unless you ran “make clean” previously. The config.nice is handy as it is a record of your config options allowing you to easily make updates and recompile. So now try the “/usr/local/apache2/bin/httpd -l” command again and you should now see mod_proxy (plus some others).
3) Install Tomcat
Download the binary tarball into your /usr/local/src/ directory using “curl -O”. The current version is 6.0.18 – http://apache.inetbridge.net/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.tar.gz
Once you have this, move it into the proper place:
# tar -xzvf apache-tomcat-6.0.18.tar.gz
# mv apache-tomcat-6.0.18 /usr/local/
# ln -s /usr/local/apache-tomcat-6.0.18/ /usr/local/tomcat
As I mentioned in a previous tutorial, I like to leave the version info intact and symlink from a generic name. This allows me to have different versions of tomcat installed in the same location and switch between them by changing the symlink. We now need to set the JAVA_HOME environment variable. This is most easily done in /etc/bashrc by adding the line:
export JAVA_HOME="/Library/Java/Home"
to the end of the file.
So now check to make sure Tomcat is working by executing the startup script /usr/local/tomcat/bin/startup.sh. The default port is 8080 so fire up a browser and check http://localhost:8080/. You should see a page stating you set up Tomcat successfully.
4) Secure Tomcat
Open the file /usr/local/tomcat/conf/tomcat-users.xml and uncomment everything between “tomcat-users”. You will need to add two more roles and at least change the default passwords. The “tomcat-users” stanza should look something like this:
<tomcat-users>
<role rolename="tomcat"/>
<role rolename="role1"/>
<role rolename="manager"/>
<role rolename="admin"/>
<user username="tomcat" password="{password}" roles="tomcat,admin,manager"/>
<user username="role1" password="{password}" roles="role1"/>
<user username="both" password="{password}" roles="tomcat,role1"/>
</tomcat-users>
I believe this change will be picked up by Tomcat, but just in case, from the Tomcat bin directory run “shutdown.sh” then “startup.sh”. You should now be able to access the Admin links on the default page at localhost:8080.
5) Configure Apache
In your Apache config file – /usr/local/apache2/conf/httpd.conf – create a VirtualHost stanza for the hostname to pass on to Tomcat. Here is an example for the hostname tomcat.latko.org:
<VirtualHost tomcat.latko.org:80>
ServerName tomcat.latko.org
ServerAlias tomcat.latko.org
ServerAdmin chris [at] latko [dot] org
DocumentRoot /Library/WebServer/Documents/tomcat
#APP
ProxyPass / http://127.0.0.1:8080/app
ProxyPassReverse / http://127.0.0.1:8080/app
#MANAGER
ProxyPass /manager http://127.0.0.1:8080/manager/
ProxyPassReverse /manager http://127.0.0.1:8080/manager/
RewriteEngine On
RewriteRule ^(.*)$ http://127.0.0.1:8080/app/$1 [P,L]
</VirtualHost>
The app is the specific application you want running at this hostname. If there is no specific app, you can remove it from the arguments so
ProxyPass / http://127.0.0.1:8080/app
becomes
ProxyPass / http://127.0.0.1:8080/
For further security, you should comment out both manager lines, making it inaccessible to anyone. Now going to http://tomcat.latko.org will take you directly to your Tomcat app or to the Tomcat root we saw at http://localhost:8080.
6) Tomcat Startup Configuration
You can increase the memory allocated to Tomcat by modifying the startup.sh script and adding the following line at the top:
JAVA_OPTS="-Xms128m -Xmx512m"
To have Tomcat automatically start on boot, create a file in /Library/LaunchDaemons/ called “org.apache.tomcat.plist” with the contents:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Disabled</key>
<false/>
<key>Label</key>
<string>com.apache.tomcat</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/tomcat/bin/startup.sh</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>
If you have any questions about this process, I would be happy to answer them.
Popularity: unranked [?]
Update: Mike Pinkerton, Mac Chrome developer, is writing some interesting tidbits on his blog.
Without being involved with Google or Chromium in any way, these are just stabs in the dark. I would love to hear from those in the know about what is going on.
As I see it, there are three fundamental reasons for the delay:
1) Priorities
Google knows the fastest way to gain market share is to bundle the browser at the OEM stage with the dominant operating system. To do this, they need a compelling case for why the browser should be bundled. This led to several things:
This raises the advertising questions:
So can’t Google just use it’s advertising space to drive adoption?
Yes, but it generally doesn’t work. Chrome made a brief showing on the Google frontpage, breaking the rule of 28 but we know this doesn’t drive much traffic. Visitors to Google are usually looking to search for something, not download a new browser. Evidence of this can be see in ReadWriteWeb’s analysis of the traffic the HTC G1 phone got from a Google link.
Also, you do see adsense ads for Chrome (instead of Firefox) popping up around IE or Firefox related articles. But I don’t think these are driving too much traffic.
2) Technology
I’m sure Chrome is packed chock full o’ technology that I don’t bring up here, but this should be a start:
WinHTTP
With the preview release of Chrome 2.0, we saw Google dump WinHTTP in favor of its own codebase. This is great and all, but leads me to believe that the original codebase was developed in a quick and dirty style in 20% time with MS tools. More on this can be read in this thread where people have combed the codebase and saw how dirty it is.
Sandboxing
This is another area where MS rears its ugly head. According to Nicolas Sylvain, Chrome developer:
The Mac and Linux version of Google Chrome are still in development. They are not ready yet.
We haven’t decided the implementation details of the sandbox on these platforms, but we clearly want something equivalent.
October 2, 2008 3:48 PM
This was in a comment made on this blog post titled “A new approach to browser security: the Google Chrome Sandbox“. It seems this is one of the main bottlenecks.
V8
I thought V8 was the culprit for the longest time until I saw this article – “Building and compiling V8 on Mac OS X“.
3) Look and Feel
These is a third and rather weak argument for the delay. Back to the “Platforms and Priorities” post, Amanda Walker, Software Engineer tells us:
One overriding goal we have had from the start has been to build the best browser we can. When it comes to Mac and Linux versions, this means that our goal is not to just “port” a Windows application to these other platforms–rather, our goal is to deliver Chromium’s innovative, Google-style user interface without rough edges on any of them.
Yes, making Chrome feel native to the Mac is important, but does that take six months? I’m not buying it.
image: Google
Popularity: unranked [?]
Quick personal post, see I’m not just a geek.
I just found this site that Owen Nieder put together for the old Del Mar Skateboard Ranch and it brought back a ton of memories. I’m proud to be included in the list of The Last Great Ones as this was a huge part of my life when I was growing up. When the park closed, I thought it was the end of the world.
I’ll contribute a story or two – maybe about the time I made a huge fireball with Cremora or when I broke my wrist and kept skating for the rest of the day or the last night the park was open – this stuff should not be forgotten.
image: Pushead
Popularity: unranked [?]
Interested in all things tech. Apple, iPhone, OSX, Xcode, LAMP, Obj-C, Cappuccino, Atlas, Sproutcore, JavaScript, Ruby, Python, GNU/Linux.