Zoetrope

If you want to access your Amazon S3 account, take a look at CL-S3 from Sven Van Caekenberghe. It's simpler than wrestling with curl and the Amazon webservices authentication ;-)

What you need

CL-S3 has the following dependencies, most of which are other packages from Sven, and most are installable using ASDF-INSTALL

• s-http-client, to speak the HTTP lingo
• s-xml, no webservices without XML right?
• s-utils, some utility code that CL-S3 uses
• s-base64, al your base are belong to us!
• KPAX, Sven's web framework (for URI-encoding functions) the latest version does not need KPAX
• ironclad, Nathan Froyds nice cryptography package, providing MD5 functionality

Download CL-S3 from Amazon S3 itself and symlink its cl-s3.asd in your ASDF systems directory.

So fire up your favourite REPL (I'm using Lispworks Personal Edition for this example) and load it al up using ASDF:

CL-USER 3 > (setf (logical-pathname-translations "home")
              `(("**;*.*.*" ,(concatenate 'string (namestring (user-homedir-pathname)) "**/*.*"))))
               (("**;*.*.*" "/Users/tarkin/**/*.*"))

CL-USER 4 > (unless (member :asdf *features*)
              (load #p"home:Lisp;asdf;init-asdf"))
; Loading text file /Users/tarkin/lisp/asdf/init-asdf.lisp
;  Loading text file /Users/tarkin/lisp/asdf/asdf.lisp
;Pushed #P"/Users/tarkin/lisp/asdf/systems/" onto ASDF central registry
#P"/Users/tarkin/lisp/asdf/init-asdf.lisp"

CL-USER 5 > (asdf:operate 'asdf:load-op :cl-s3)
; loading system definition from /Users/tarkin/lisp/asdf/systems/cl-s3.asd into
...
; Loading fasl file /Users/tarkin/Lisp/cl-s3/cl-s3-package.nfasl
; Loading fasl file /Users/tarkin/Lisp/cl-s3/cl-s3.nfasl
NIL

Giving CL-S3 the lowdown on your account

Set up your Amazon webservices access identifiers (access key id and secret access key id)

CL-S3 7 > (setf *access-key-id* "AAAAYKR2D401BBB09CCC")
"AAAAYKR2D401BBB09CCC"

CL-S3 4 > (setf *secret-access-key* "ooh_my_secret_is_very_secret")
"ooh_my_secret_is_very_secret"

Doing the deed

So now we should be able to query the service using CL-S3.

CL-S3 11 > (get-service)
;; CL-S3 GET http://s3.amazonaws.com/
((|ListAllMyBucketsResult| :|xmlns| "http://s3.amazonaws.com/doc/2006-03-01/") 
 (|Owner| (ID "beeec79bbda42e62b7ac74273effdabef1ddadf1b22b4c683cc91bf64d62f033") 
          (|DisplayName| "nickypeeters")) 
 (|Buckets| 
  (|Bucket| 
   (|Name| "lisp") 
   (|CreationDate| "2007-05-29T13:46:12.000Z"))))
200
((:X-AMZ-ID-2 . "wb2EU2Etuv8Mw6+Y9jonKPm6lf0uGMxfcspgwnKK0YRCYorteJ08FH1cmD5ZhtpU") 
 (:X-AMZ-REQUEST-ID . "07C15D88FD3EB8CD") (:DATE . "Tue, 29 May 2007 17:51:23 GMT") 
 (:CONTENT-TYPE . "application/xml") 
 (:TRANSFER-ENCODING . "chunked") 
 (:SERVER . "AmazonS3"))
#<URI http://s3.amazonaws.com:80>
:KEEP-ALIVE

Let's make a bucket to hold our objects in using put-bucket

CL-S3 15 > (put-bucket "zoetrope")
;; CL-S3 PUT http://s3.amazonaws.com/zoetrope
""
200
((:X-AMZ-ID-2 . "Dxl2qxeYXo6VvuOBscjnWEQaPZjaQlk97E+XLsL+DVRVmQMafEaRrpEnFrWyf5k9") 
 (:X-AMZ-REQUEST-ID . "BE93305AD58B1777") 
 (:DATE . "Tue, 29 May 2007 18:01:36 GMT") 
 (:LOCATION . "/zoetrope") 
 (:CONTENT-LENGTH . "0") 
 (:SERVER . "AmazonS3"))
#<URI http://s3.amazonaws.com:80/zoetrope>
:NEW

Put a little text under the new bucket using put-object. You need to provide a key (e.g. filename) and the content-type of the object (e.g. text/plain). S3 returns also returns the uri of the new object.

CL-S3 16 > (put-object "zoetrope" 
            "practical-common-lisp" 
            "Practical Common Lisp is a nice book on Common Lisp" 
            "text/plain")
;; CL-S3 PUT http://s3.amazonaws.com/zoetrope/practical-common-lisp
""
200
...
#<URI http://s3.amazonaws.com:80/zoetrope/practical-common-lisp>
:NEW

So lets see if Amazon S3 really has our little text at http://s3.amazonaws.com:80/zoetrope/practical-common-lisp using an s-http-client request

CL-S3 17 > (s-http-client:do-http-request "http://s3.amazonaws.com:80/zoetrope/practical-common-lisp")
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Error>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message><RequestId>4BA0925CDE61CC24</RequestId>
  <HostId>T2NKSHtx1nOaLcbcllgy62wPlDP/GPp//MecPlcORke/gFSE7M2zpVEevGtlmIGq</HostId>
</Error>"
403
...

Oops ! S3 returns an AccessDenied error in XML. By default any object you put is for-your-eyes-only. So let's make an authenticated request using get-object

CL-S3 20 > (get-object "zoetrope" "practical-common-lisp")
;; CL-S3 GET http://s3.amazonaws.com/zoetrope/practical-common-lisp
"Practical Common Lisp is a nice book on Common Lisp"
200
...

Uploading a file and making it public

CL-S3 doesn't have a file-upload utility method, so let's use a quick slurp-file function. We set the object to public and read-only by using the canned access-policy with the x-amz-acl header of public-read

CL-S3 21 > (defun contents-of-file (pathname)
  (with-output-to-string (contents)
    (with-open-file (in pathname :direction :input)
      (s-utils:copy-stream in contents))))

CL-S3 22 > (put-object "zoetrope" "lispification.pdf" 
             (contents-of-file "/Users/tarkin/lispification.pdf") 
             "application/pdf" 
             :amz-headers '(("x-amz-acl" . "public-read")))
;; CL-S3 PUT http://s3.amazonaws.com/zoetrope/lispification.pdf
""
200
...
#<URI http://s3.amazonaws.com:80/zoetrope/lispification.pdf>
:KEEP-ALIVE

CL-S3 has 2 utility functions for downloading and uploading files

CL-S3 39 > (download-file "lisp" "lispification.pdf")
;; CL-S3 GET http://s3.amazonaws.com/lisp/lispification.pdf
#<STREAM::LATIN-1-FILE-STREAM /Users/tarkin/lispification.pdf>
200
...
#<URI http://s3.amazonaws.com:80/lisp/lispification.pdf>
:KEEP-ALIVE

You can see the function returns the file-stream where the file was written. You can optionally give a directory to save the file in. Uploading a file works in the same way.

CL-S3 40 > (upload-file "/Users/tarkin/Lisp/init_lispworks.lisp" "lisp" 
         :mime-type "text/plain" :acl "public-read")
;; CL-S3 PUT http://s3.amazonaws.com/lisp/init_lispworks.lisp
""
200
...
#<URI http://s3.amazonaws.com:80/lisp/init_lispworks.lisp>
:KEEP-ALIVE

Check out the CL-S3 API docs for more info.