How To Determine the Size of an S3 Bucket

This one came up while working on my home network / photo management setup. I’ve set my Synology DS216+ NAS to use Cloud Sync to back up my files to an Amazon S3 bucket (see this post for some more information on using S3 for backups). The problem was it was taking a very long time, and I needed to figure out how much had transferred.

Unfortunately, Amazon has no simple mechanism for determining the size of an S3 bucket. I found a couple posts on StackOverflow showing how to do it, but they seemed overly complex.

While you can get a bucket size using several third party GUI tools, the command line approach is quick and easy. It does require the Amazon Command line Tools to be installed, and access keys generated, but once that’s done, you can quickly query Amazon for just about anything.

Here’s the command I used to determine the size of my bucket. This is on a mac:

$ aws s3 ls s3://BUCKETNAME --recursive | awk '{total+=$3} END{print "total =",total/1024/1024," MB"}'

This will give back something like:

total = 245032  MB

Voila! Time for that command can vary depending on the size of the bucket. For me, with around 20,000 photos stored, it takes about 20 seconds.

Wherein Dave learns that Podcasts are Kinda Awesome

Face it, I’ve been on the net a long time. Usually by the time some buzzword I’ve heard gets enough attention that I check into it, it turns out the hype doesn’t even remotely match the reality. And thus it’s been for me with Podcasts. You can’t swing a dead cat without hearing someone say “And subscribe to our podcast!!!”

Maybe it’s just my early trauma dealing with downloaded files / iTunes syncing problems / PalmPilot lack of audio, whatever, but I never listened to podcasts, even when a friend would say “Hey, did you hear that podcast by Bubbitah Bingah? Dude was awesome.” “Uh huh, what’s the link to the text of the article again?”

Well, even old dogs can learn new tricks. I’m into the third year of dealing with a 40 minute commute, all along highways, with nothing but XM radio or NPR to listen to, I decided to finally take the plunge and check out podcasts. Herein lies what I’ve learned.

Get a good a podcast app

First things first. You need a way to listen to podcasts. For me the important mechanism would be something that allows for downloads ahead of time (say, over Wifi), and then I could catch up as the week went along. I’m fully wedded to Android now, happily using my Motorola Moto X for gaming, music, mail, calendaring – heck, everything. So a decent Android app was needed. I settled on Podcast Addict. It’s a great app that categorizes all my subscriptions, and lets me download all or some of the episodes ahead of time.

Now all I needed was content

So, what do you want to listen to?

There’s seriously no shortage of podcasts out there. And, frankly, most of them suck. When you have no need to limit your time to a 10 minute slot on a radio show, and can blather on for an hour and a half about navel lint, the field gets crowded pretty quickly.

I recommend starting with things you know – being an NPR addict, these were easy:

RadioLab
Wait Wait Don’t Tell Me
Marketplace
CarTalk
This American Life

But after that, we start getting into things that are related, but you don’t get to hear quite as often.

The Moth
Ted Radio Hour

And then we get into fun stuff. Up until this point, all of that stuff I’d hear on the radio from time to time (either on NPR or on Public Radio Remix – a station I highly recommend, btw). But what about independent stuff? This is what I came to share with you.

First, I have to highly highly recommend Our Fake History. This podcast focuses on deep dives into historical myths, legends, and stories, and digs out what parts of those stories are true and what has been embellished over time. I got completely sucked into their first ‘big’ series, “Was There Really a Trojan War”. I learned more about the Iliad, Greek mythology, and 19th century archeology than I had ever known before. The current series is going into Helen of Troy, and it’s equally fascinating. Highly recommended!

Following along after is Lore. In a similar vein, this podcast talks about history with a sort of dark bent. Vampires, missing persons – where did all these stories come from? It has a darker, more ‘sitting around the fireplace telling stories’ feel, but all of it is well researched and detailed.

Moving off the dark history bit, I also listen to the Petapixel podcast. This series follows the website pretty closely, but has extra commentary and thoughts by Mike “Sharky” James. Great stuff.

One last shout out. My friend Tim pointed me to Welcome to Night Vale. This is the fictional broadcast of a public service radio station in the town of Night Vale. Think of it as a sort of Prairie Home Companion meets HP Lovecraft. It has it’s ups and downs, but has some great moments in it.

So, if you’ve ever thought about fiddling around with podcasts, I recommend getting Podcast Addict (it’s free), anv giving these podcasts a try. There’s lots to be learned.