Feed on
Posts
Comments

JetS3t 0.7.4

The latest release of JetS3t, version 0.7.4, was released over the weekend.

Bug fixes:

  • Shell scripts are more compatible with Cygwin

New toolkit functionality:

  • Added support for the new Reduced Redundancy Storage (RRS) class for objects
  • CloudFrontService: Support for HTTPS-only distributions and logging of streaming distributions
  • Added support for buckets located in the Asia Pacific (Singapore) location “ap-southeast-1″
  • Improved compatibility with Eucalyptus/Walrus

Cockpit application updates:

  • Reduced Redundancy Storage class
  • Buckets located in the Asia Pacific (Singapore) location
  • HTTPS-only CloudFront distributions

Synchronize application updates:

  • Allow synchronization with third-party buckets that are not owned by the user

Other notes:

  • CloudFrontService API changes may break backwards-compatibility
  • SOAPService is deprecated and will soon be removed from the toolkit

Visit the JetS3t web site to download the latest release and view the latest code samples and API Javadoc. The latest version should also be available from the official Maven2 repository by now.

Read about the complete list of changes in the release notes, and visit the development site to submit bug reports or help out with the project.

As of a few minutes ago the latest JetS3t code includes support for a new S3 feature called Reduced Redundancy Storage. If you keep non-vital data in S3 you can now choose to accept “reduced redundancy” for this data in return for a cheaper storage rate.

Accept a little more risk, save some bucks. For many S3 customers this will be an attractive option.

See Amazon’s RRS blog post for more information.

Below is a brief overview of how you can use the feature in JetS3t once you have downloaded and built the latest code.

Cockpit

The Cockpit application has support for the new storage class in a few places:

  • In the preferences you can choose the default storage class to apply when uploading objects
  • the Copy or Move Objects dialog allows you to choose the storage class for destination objects. This makes it easy to apply the cheaper storage option to many objects, you simply copy the objects in-place after selected the REDUCED_REDUNDANCY storage class
  • each object’s current storage class is now shown in the Object Attributes dialog.

API

In the API you set the storage class of an object prior to uploading it:

// Create an object as usual
S3Object object = new S3Object("my-object", "some data");
// Set the non-default storage class prior to upload
object.setStorageClass(
    S3Object.STORAGE_CLASS_REDUCED_REDUNDANCY);
// Upload as usual
s3Service.putObject("bucket-name", object);

To apply the new storage class to existing objects you call the normal RestS3Service copyObject or moveObject methods after setting the storage class attribute of each destination object.

To check the storage class an object is stored under you perform a bucket listing then call the S3Object’s getStorageClass method:

S3Object[] objects = s3Service.listObjects("bucket-name");
for (int i = 0; i < objects.length; i++) {
    System.out.println(objects[i].getKey() + ": "
        + objects[i].getStorageClass());
}

Amazon has announced the new Access Logs feature for CloudFront streaming distributions. With access logging turned on you can see when and how people interact with your streamed content.

The latest JetS3t code provides basic API support for enabling and managing these access logs, in much the same way as you do for non-streaming distributions. See the recent changes to the CloudFrontSamples.java file for example usage.

Note that this new feature has not yet been widely tested so file a bug report to let me know if you have problems.

Ratings Pollution

I’m thinking of buying Michael Lewis’ new book, The Big Short. So I visit the Amazon site and see a pretty terrible average review of only 2.5 stars. What gives?

Turns out a bunch of douches have given the book one-star reviews because the book is not yet available on the Kindle.

Not only are they submitting low reviews because they cannot yet buy the book in their preferred form — a decision that is most likely made by the publisher not the author — but they are doing so without even reading the book. This is a whiny and pathetic form of “protest” that does nothing but pollute the digital commons. Grow up folks!

JetS3t 0.7.3

I have just released JetS3t version 0.7.3 following the announcement earlier this week that S3 Versioning has graduated to production status.

This release includes:

Visit the JetS3t web site to download the latest release and view the latest code samples and API Javadoc. The official Maven2 repository should be updated in the next day or so.

Read about the complete list of changes in the release notes, and visit the development site to submit bug reports or contribute to the project.

There is a lot of S3 and JetS3t news tonight.

Versioning For All

To begin with, the new S3 beta versioning feature is now available in all regions. This means that you can retain past versions of all your S3 objects regardless of where your bucket is located.

The latest JetS3t code has full support for versioning that makes it very easy to use. You can enable versioning for a bucket like so:

restS3Service.enableBucketVersioning("bucket-name");

Then should you ever need to recover some data — such as after accidentally deleting an object or overwriting data with a corrupted file — you can find and retrieve the prior versions:

// List an object's prior versions
BaseVersionOrDeleteMarker[] versions = restS3Service
    .getObjectVersions("bucket-name", "object-name");

// Retrieve the next-to-last version of data
String versionId = versions[versions.length - 2].getVersionId();
S3Object priorVersionObject = s3Service.getVersionedObject(
    versionId, "bucket-name", "object-name");

The Second Factor

As well as rolling out broader availability of the versioning feature Amazon has (somewhat quietly) added another interesting feature: the first API-level support for multi-factor authentication. Multi-factor authentication (MFA) adds an extra level of security to systems by requiring users to prove ownership of a token or device of some kind in addition to their normal login credentials. This means that even if someone steals or guesses your credentials they will be unable to perform actions on your account because they do not possess the device.

In Amazon’s case, like PayPal and some banks before them, the additional factor comprises a small electronic device that generates code numbers. Once you have purchased one of these devices and enabled it in your AWS account you will be required to provide an extra code number when performing certain tasks.

Previously the additional MFA device code was only required when you logged in to the AWS Console but as of today you can turn on MFA for your S3 buckets in tandem with versioning. When versioning with MFA is enabled not only will the bucket’s owner be the only user who can permanently delete object versions, but this user will be required to provide a time-limited MFA code to do so.

Again, this is relatively straight-forward to use in JetS3t:

// Require MFA to permanently delete object versions
restS3Service.enableBucketVersioningWithMFA("bucket-name");

// Obtain user's MFA device serial number and time-limited code
String multiFactorSerialNumber = "#111222333";
String multiFactorAuthCode = "12345678";

// Delete an MFA-protected object version
restS3Service.deleteVersionedObjectWithMFA(versionId,
    multiFactorSerialNumber, multiFactorAuthCode,
    "bucket-name", "object-name");

The addition of MFA support at the API level in S3 is particularly interesting because this is the first time Amazon has done so, and because it raises some interesting challenges for developers who are accustomed to building fully-automated systems. To take advantage of the protection the MFA provides a system will need to prompt the user for her MFA code every 30 seconds or so when she wishes to permanently delete data. I am keen to see how — and if — developers actually build this feature into their applications.

Hello BitBucket

Finally, repeating the news I posted recently on the JetS3t discussion forums, I have decided to move the JetS3t codebase from it’s old home at java.net over to the BitBucket service: http://bitbucket.org/jmurty/jets3t/

BitBucket has the advantage of being a more modern, easy-to-navigate site, and has seamless support for Mercurial which is my favorite source code management tool. So it’s farewell to java.net and CVS, you served us well but it’s time for some new blood.

Try out the latest code and let me know what you think. Head over to the JetS3t BitBucket repository and grab the latest code via a pull (if you’re familiar with Mercurial) or simply download it via the “get source” link.

Older Posts »