Subscribe to MAKE Magazine Today!

• Coding plugins for Internet Explorer
• Main
• HOWTO - iPod and PSP movies on Windows Mobile

Wikipedia offline reader: put all of Wikipedia on your laptop

Wikipedia periodically publishes full data dumps of the encyclopedia's content. If you wanted to make your own copy of the Wikipedia site for offline viewing, you'd typically convert and import that content into MySQL using MediaWiki's importDump.php utility. The initial import process can take over a day. Building the indexes for searching articles takes even longer.

Thanassis Tsiodras came up with a better way of using the Wikipedia dump for offline reading:

Wouldn't it be perfect, if we could use the wikipedia "dump" data JUST as they arive after the download? Without creating a much larger (space-wize) MySQL database? And also be able to search for parts of title names and get back lists of titles with "similarity percentages"?

The end result is a Wikipedia reader that indexes the entire dump in under 30 minutes, stores the entire data in the original, though segmented, bz2 compressed format, and comes complete with a light web interface for searching and reading entries.

Building a (fast) Wikipedia offline reader - Link

Related:
WikipediaFS - a Linux MediaWiki file-system - Link

Posted by Jason Striegel | Aug 19, 2007 10:05 PM
Web | Permalink | Comments (3)

Recent Entries

• Poromenos' hello world curve
• USB CapsLocker and Sun keyboard simulation
• Robosapien has a coil gun
• Faster Windows shutdown
• Assign USB drives to a folder
• Little drummer bot
• CSS ad blocking for Firefox and Safari
• Design Coding: web standards rap
• Shredz64: Guitar Hero for C64
• BATMAN: adhoc mesh routing

Bloggers

Welcome to the Hacks Blog!