{"metadata":{"image":[],"title":"","description":""},"api":{"url":"","auth":"required","params":[],"results":{"codes":[]},"settings":""},"next":{"description":"","pages":[]},"title":"Item Bulk Upload","type":"basic","slug":"bulk-upload","excerpt":"","body":"Most archive.org users upload items through the web interface. This option isn't ideal for supporting users who need to upload many items or files at a time. The Internet Archive provides 2 mechanisms for uploading items in bulk. \n[block:api-header]\n{\n \"type\": \"basic\",\n \"title\": \"IAS3 (Experts)\"\n}\n[/block]\nThe first option is to use the ias3 service directly. IAS3 is the Internet Archive's implementation of Amazon's AWS S3 (Simple Storage Service). You can read more about it here <link>. IAS3 provides an API for creating and uploading archive.org items programatically but it requires a high level of technical proficiency to use.\n\nSee common examples at: https://github.com/vmbrasseur/IAS3API/blob/master/examples/README.md\n[block:code]\n{\n \"codes\": [\n {\n \"code\": \"curl --location --header 'x-amz-auto-make-bucket:1' \\\\\\n --header 'x-archive-meta01-collection:opensource' \\\\\\n --header 'x-archive-meta-mediatype:texts' \\\\\\n --header 'x-archive-meta-sponsor:Andrew W. Mellon Foundation' \\\\\\n --header 'x-archive-meta-language:eng' \\\\\\n --header \\\"authorization: LOW $accesskey:$secret\\\" \\\\\\n --upload-file /home/samuel/public_html/intro-to-k.pdf \\\\\\n http://s3.us.archive.org/sam-s3-test-08/demo-intro-to-k.pdf\",\n \"language\": \"curl\",\n \"name\": \"upload\"\n }\n ]\n}\n[/block]\n\n[block:api-header]\n{\n \"type\": \"basic\",\n \"title\": \"The `internetarchive` Client Library (Everyone)\"\n}\n[/block]\nAnother option is to use the `internetarchive` Python client library. This library includes a command line utility called the \"ia\" tool which wraps IAS3 and makes it easy for anyone to upload in bulk.\n\nThe easiest way to get started, is to use a binary. The only requirements for using a binary are that you have Python 2 or 3 installed in a unix-like environment.\n\nOnce you have [downloaded a binary of ia](http://internetarchive.readthedocs.io/en/latest/installation.html#binaries) and [configured it with your archive.org credentials](http://internetarchive.readthedocs.io/en/latest/quickstart.html#configuring), take some time to familiarize yourself with [what an archive.org item is and what limitations items have](http://internetarchive.readthedocs.io/en/latest/items.html). Then follow these instructions for bulk uploading: http://internetarchive.readthedocs.io/en/latest/cli.html#bulk-uploading.\n\nPlease see http://internetarchive.readthedocs.io/en/latest/metadata.html for more information on item metadata.","updates":["5c4857f2485fc60ff058ada9"],"order":5,"isReference":false,"hidden":false,"sync_unique":"","link_url":"","link_external":false,"_id":"56d553b3c8c687150071efc0","category":{"sync":{"isSync":false,"url":""},"pages":["56d553b3c8c687150071efc0","56d553dc075dfa0b00be98f3","56d5582aa4a9211b00c8f21b","56d819277a04df0b00ddf058","56d81a477a04df0b00ddf05c","56d81da4c83c5f0b00d28719"],"title":"Archive.org","slug":"archiveorg","order":1,"from_sync":false,"reference":false,"_id":"56d5537ac8c687150071efbf","__v":6,"createdAt":"2016-03-01T08:31:54.133Z","project":"56d55013075dfa0b00be98e6","version":"56d55014075dfa0b00be98e9"},"githubsync":"","project":"56d55013075dfa0b00be98e6","user":"56d54fd3c8c687150071efb0","createdAt":"2016-03-01T08:32:51.028Z","parentDoc":null,"version":{"version":"1","version_clean":"1.0.0","codename":"Archy","is_stable":true,"is_beta":false,"is_hidden":false,"is_deprecated":false,"categories":["56d55014075dfa0b00be98ea","56d55036c8c687150071efb3","56d55094b939870b005129e2","56d550ae5acf9f0b00f735d0","56d5537ac8c687150071efbf","573cae191000320e00918252","573cbe15f320ae29009dbd94","5792a0baf967f617000d9e77","5792a0c023106419009c4302","58093f2994bc150f00f2fe47","580940f33c22790f00fd1e98","58094c01e5998d0f00d29b5d","58101442c03f0b0f0097f099","581020f07c59d72500b85add","581032c708d8050f00948d2a","5810e6ca08d8050f00948dcf"],"_id":"56d55014075dfa0b00be98e9","__v":16,"createdAt":"2016-03-01T08:17:24.051Z","project":"56d55013075dfa0b00be98e6","releaseDate":"2016-03-01T08:17:24.051Z"},"__v":3}