{"_id":"56d553b3c8c687150071efc0","category":{"_id":"56d5537ac8c687150071efbf","__v":6,"pages":["56d553b3c8c687150071efc0","56d553dc075dfa0b00be98f3","56d5582aa4a9211b00c8f21b","56d819277a04df0b00ddf058","56d81a477a04df0b00ddf05c","56d81da4c83c5f0b00d28719"],"project":"56d55013075dfa0b00be98e6","version":"56d55014075dfa0b00be98e9","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-03-01T08:31:54.133Z","from_sync":false,"order":1,"slug":"archiveorg","title":"Archive.org"},"project":"56d55013075dfa0b00be98e6","user":"56d54fd3c8c687150071efb0","parentDoc":null,"version":{"_id":"56d55014075dfa0b00be98e9","__v":16,"project":"56d55013075dfa0b00be98e6","createdAt":"2016-03-01T08:17:24.051Z","releaseDate":"2016-03-01T08:17:24.051Z","categories":["56d55014075dfa0b00be98ea","56d55036c8c687150071efb3","56d55094b939870b005129e2","56d550ae5acf9f0b00f735d0","56d5537ac8c687150071efbf","573cae191000320e00918252","573cbe15f320ae29009dbd94","5792a0baf967f617000d9e77","5792a0c023106419009c4302","58093f2994bc150f00f2fe47","580940f33c22790f00fd1e98","58094c01e5998d0f00d29b5d","58101442c03f0b0f0097f099","581020f07c59d72500b85add","581032c708d8050f00948d2a","5810e6ca08d8050f00948dcf"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"Archy","version_clean":"1.0.0","version":"1"},"__v":2,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-03-01T08:32:51.028Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":5,"body":"Most archive.org users upload items through the web interface. This option isn't ideal for supporting users who need to upload many items or files at a time. The Internet Archive provides 2 mechanisms for uploading items in bulk. \n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"IAS3 (Experts)\"\n}\n[/block]\nThe first option is to use the ias3 service directly. IAS3 is the Internet Archive's implementation of Amazon's AWS S3 (Simple Storage Service). You can read more about it here <link>. IAS3 provides an API for creating and uploading archive.org items programatically but it requires a high level of technical proficiency to use.\n\nSee common examples at: https://github.com/vmbrasseur/IAS3API/blob/master/examples/README.md\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl --location --header 'x-amz-auto-make-bucket:1' \\\\\\n     --header 'x-archive-meta01-collection:opensource' \\\\\\n     --header 'x-archive-meta-mediatype:texts' \\\\\\n     --header 'x-archive-meta-sponsor:Andrew W. Mellon Foundation' \\\\\\n     --header 'x-archive-meta-language:eng' \\\\\\n     --header \\\"authorization: LOW $accesskey:$secret\\\" \\\\\\n     --upload-file /home/samuel/public_html/intro-to-k.pdf \\\\\\n     http://s3.us.archive.org/sam-s3-test-08/demo-intro-to-k.pdf\",\n      \"language\": \"curl\",\n      \"name\": \"upload\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"The `internetarchive` Client Library (Everyone)\"\n}\n[/block]\nAnother option is to use the `internetarchive` Python client library. This library includes a command line utility called the \"ia\" tool which wraps IAS3 and makes it easy for anyone to upload in bulk.\n\nThe easiest way to get started, is to use a binary. The only requirements for using a binary are that you have Python 2 or 3 installed in a unix-like environment.\n\nOnce you have [downloaded a binary of ia](http://internetarchive.readthedocs.io/en/latest/installation.html#binaries) and [configured it with your archive.org credentials](http://internetarchive.readthedocs.io/en/latest/quickstart.html#configuring), take some time to familiarize yourself with [what an archive.org item is and what limitations items have](http://internetarchive.readthedocs.io/en/latest/items.html). Then follow these instructions for bulk uploading: http://internetarchive.readthedocs.io/en/latest/cli.html#bulk-uploading.\n\nPlease see http://internetarchive.readthedocs.io/en/latest/metadata.html for more information on item metadata.","excerpt":"","slug":"bulk-upload","type":"basic","title":"Item Bulk Upload"}