{"metadata":{"image":[],"title":"","description":""},"api":{"url":"","auth":"required","params":[],"results":{"codes":[]},"settings":""},"next":{"description":"","pages":[]},"title":"Item Bulk Upload","type":"basic","slug":"bulk-upload","excerpt":"","body":"Most archive.org users upload items through the web interface. This option isn't ideal for supporting users who need to upload many items or files at a time. The Internet Archive provides 2 mechanisms for uploading items in bulk. \n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"IAS3 (Experts)\"\n}\n[/block]\nThe first option is to use the ias3 service directly. IAS3 is the Internet Archive's implementation of Amazon's AWS S3 (Simple Storage Service). You can read more about it here <link>. IAS3 provides an API for creating and uploading archive.org items programatically but it requires a high level of technical proficiency to use.\n\nSee common examples at: https://github.com/vmbrasseur/IAS3API/blob/master/examples/README.md\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl --location --header 'x-amz-auto-make-bucket:1' \\\\\\n     --header 'x-archive-meta01-collection:opensource' \\\\\\n     --header 'x-archive-meta-mediatype:texts' \\\\\\n     --header 'x-archive-meta-sponsor:Andrew W. Mellon Foundation' \\\\\\n     --header 'x-archive-meta-language:eng' \\\\\\n     --header \\\"authorization: LOW $accesskey:$secret\\\" \\\\\\n     --upload-file /home/samuel/public_html/intro-to-k.pdf \\\\\\n     http://s3.us.archive.org/sam-s3-test-08/demo-intro-to-k.pdf\",\n      \"language\": \"curl\",\n      \"name\": \"upload\"\n    }\n  ]\n}\n[/block]\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"The `internetarchive` Client Library (Everyone)\"\n}\n[/block]\nAnother option is to use the `internetarchive` Python client library. This library includes a command line utility called the \"ia\" tool which wraps IAS3 and makes it easy for anyone to upload in bulk.\n\nThe easiest way to get started, is to use a binary. The only requirements for using a binary are that you have Python 2 or 3 installed in a unix-like environment.\n\nOnce you have [downloaded a binary of ia](http://internetarchive.readthedocs.io/en/latest/installation.html#binaries) and [configured it with your archive.org credentials](http://internetarchive.readthedocs.io/en/latest/quickstart.html#configuring), take some time to familiarize yourself with [what an archive.org item is and what limitations items have](http://internetarchive.readthedocs.io/en/latest/items.html). Then follow these instructions for bulk uploading: http://internetarchive.readthedocs.io/en/latest/cli.html#bulk-uploading.\n\nPlease see http://internetarchive.readthedocs.io/en/latest/metadata.html for more information on item metadata.","updates":["5c4857f2485fc60ff058ada9","617a31955bb1cd006af50c55"],"order":5,"isReference":false,"hidden":false,"sync_unique":"","link_url":"","link_external":false,"_id":"56d553b3c8c687150071efc0","category":{"sync":{"isSync":false,"url":""},"pages":["56d553b3c8c687150071efc0","56d553dc075dfa0b00be98f3","56d5582aa4a9211b00c8f21b","56d819277a04df0b00ddf058","56d81a477a04df0b00ddf05c","56d81da4c83c5f0b00d28719"],"title":"Archive.org","slug":"archiveorg","order":1,"from_sync":false,"reference":false,"_id":"56d5537ac8c687150071efbf","__v":6,"createdAt":"2016-03-01T08:31:54.133Z","project":"56d55013075dfa0b00be98e6","version":"56d55014075dfa0b00be98e9"},"githubsync":"","project":"56d55013075dfa0b00be98e6","user":"56d54fd3c8c687150071efb0","createdAt":"2016-03-01T08:32:51.028Z","parentDoc":null,"version":{"version":"1","version_clean":"1.0.0","codename":"Archy","is_stable":true,"is_beta":false,"is_hidden":false,"is_deprecated":false,"categories":["56d55014075dfa0b00be98ea","56d55036c8c687150071efb3","56d55094b939870b005129e2","56d550ae5acf9f0b00f735d0","56d5537ac8c687150071efbf","573cae191000320e00918252","573cbe15f320ae29009dbd94","5792a0baf967f617000d9e77","5792a0c023106419009c4302","58093f2994bc150f00f2fe47","580940f33c22790f00fd1e98","58094c01e5998d0f00d29b5d","58101442c03f0b0f0097f099","581020f07c59d72500b85add","581032c708d8050f00948d2a","5810e6ca08d8050f00948dcf"],"_id":"56d55014075dfa0b00be98e9","__v":16,"createdAt":"2016-03-01T08:17:24.051Z","project":"56d55013075dfa0b00be98e6","releaseDate":"2016-03-01T08:17:24.051Z"},"__v":4}

Introduction

Archive.org

Wayback Machine

Books

OpenLibrary

Official Client Libraries

Labs IIIF API

Labs / Experiments

Community

Item Bulk Upload

Most archive.org users upload items through the web interface. This option isn't ideal for supporting users who need to upload many items or files at a time. The Internet Archive provides 2 mechanisms for uploading items in bulk. [block:api-header] { "type": "basic", "title": "IAS3 (Experts)" } [/block] The first option is to use the ias3 service directly. IAS3 is the Internet Archive's implementation of Amazon's AWS S3 (Simple Storage Service). You can read more about it here <link>. IAS3 provides an API for creating and uploading archive.org items programatically but it requires a high level of technical proficiency to use. See common examples at: https://github.com/vmbrasseur/IAS3API/blob/master/examples/README.md [block:code] { "codes": [ { "code": "curl --location --header 'x-amz-auto-make-bucket:1' \\\n --header 'x-archive-meta01-collection:opensource' \\\n --header 'x-archive-meta-mediatype:texts' \\\n --header 'x-archive-meta-sponsor:Andrew W. Mellon Foundation' \\\n --header 'x-archive-meta-language:eng' \\\n --header \"authorization: LOW $accesskey:$secret\" \\\n --upload-file /home/samuel/public_html/intro-to-k.pdf \\\n http://s3.us.archive.org/sam-s3-test-08/demo-intro-to-k.pdf", "language": "curl", "name": "upload" } ] } [/block] [block:api-header] { "type": "basic", "title": "The `internetarchive` Client Library (Everyone)" } [/block] Another option is to use the `internetarchive` Python client library. This library includes a command line utility called the "ia" tool which wraps IAS3 and makes it easy for anyone to upload in bulk. The easiest way to get started, is to use a binary. The only requirements for using a binary are that you have Python 2 or 3 installed in a unix-like environment. Once you have [downloaded a binary of ia](http://internetarchive.readthedocs.io/en/latest/installation.html#binaries) and [configured it with your archive.org credentials](http://internetarchive.readthedocs.io/en/latest/quickstart.html#configuring), take some time to familiarize yourself with [what an archive.org item is and what limitations items have](http://internetarchive.readthedocs.io/en/latest/items.html). Then follow these instructions for bulk uploading: http://internetarchive.readthedocs.io/en/latest/cli.html#bulk-uploading. Please see http://internetarchive.readthedocs.io/en/latest/metadata.html for more information on item metadata.