@mapbox/wikibush

wikibush -------- Put wikidata dumps into `rbush`. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be access

Stats

StarsIssuesVersionUpdatedCreatedSize
@mapbox/wikibush
0.0.54 years ago5 years agoMinified + gzip package size for @mapbox/wikibush in KB

Readme

wikibush

Put wikidata dumps into rbush. To get around blowing up memory of a single process, this sets up shards and datafiles for seeking on disk rather than storing full feature data in memory. Goal is to allow full wikidata dump to be accessible via a single interface.

Usage example

# stream wikidata dump into wikibush storage
mkdir workdir
cat wikidata-20161107-geo.ldjson.bz2 - | bzip2 -d | wikibush-writer --workdir=workdir --tolerance=0.001

# read features around point back out
wikibush-reader --workdir=workdir --lon=-73.9975 --lat=40.7308

If you find any bugs or have a feature request, please open an issue on github!

The npm package download data comes from npm's download counts api and package details come from npms.io.