Package of tools to interrogate London Datastore with Python
Vaguely inspired by https://github.com/Greater-London-Authority/ldndatar although won't be API-compatible for a while.
This is an extremely unstable implementation. Classes and functions are likely to change rapidly.
If you're interested in using it, please contact Sven sven.latham@london.gov.uk and we can work together!
You will need an API key to use the command-line interface (CLI) and library. The CLI expects this to be provided via the DATASTORE_API environment key.
To get a key, visit your profile on the Datastore at [https://data.london.gov.uk/user] and click Get API Key. This will show your API key. Note that keys are not regenerated by clicking on this button.
To feed the API key into the CLI, execute the following before running the CLI:
DATASTORE_API=abcdef-1234-sample
export DATASTORE_API
Get the latest version:
pip install git+ssh://git@github.com/Greater-London-Authority/lds-data-py.git
Run the CLI (basic interactions):
python cli.py -a {action} -i {input dataset} [-o {output dataset}]
Please note the test suite uses a live dataset (lds-api-test-area) which you will need permission to edit. During the test, files will be added, modified and removed at this location.
from lds_data import ldsagent
# api_key is the API key of your user from https://data.london.gov.uk/user
agent = LdsAgent(api_key)
# To get a list of resources
# dataset -> the dataset's slug (string)
resources = agent.get_resources(dataset)
# To add a new resource
# title -> the title of the file (string)
# srcpath -> the path to the file locally (string)
agent.add_resource(dataset, title, srcpath)
# To replace an existing resource
# key -> server-side identifier for the resource (obtained from get_resources)
# srcpath -> the path of the replacement file
agent.update_resource(dataset, key, srcpath)
# To sync a local folder remotely (copy local files to remote if newer)
# Note this doesn't currently delete files on the server
# dataset -> the dataset's slug
# localdir -> local directory to mirror (without closing slash)
agent.sync_dir(dataset, localdir)