Backup all spaces (a one-liner)

Clubhouse
1

So, I wanted a way to backup all my spaces. It took me a few hours, but I wrote a shell script to do it :slight_smile:

KINOPIO_API_KEY=<your key> \
curl -H "Authorization: $KINOPIO_API_KEY" https://api.kinopio.club/user/spaces | \
jq -r ".[] | .id + \",\" + (.name | tojson)" | \
awk '{ print "curl -H \"Authorization: $KINOPIO_API_KEY\" https://api.kinopio.club/space/"$1" > "$2".json" }' FS=, OFS=, | \
xargs -0 -n1 bash -c

I wrote up an explanation of it on my blog.

I’m running this hourly. The first time I ran it, I got a 401 for some API calls. @pirijan let me know if I need to tweak things lol

2 Likes
2

The most likely issue is that throttling is happening as you surmised, you can avoid the rate limit by adding a 1 second delay before each https://api.kinopio.club/space/ curl request. In javascript, I’d use setTimeout, but I’m not sure what the Bash equivalent is.

It is strange that it returned a 401 though (the rate limiter should return a 429).

If, after adding throttling, you still get 401s my recommended debug step:,

  • print the spaceid request that failed. Does it always fail?
  • PM it to me and I’ll look into that.
  • Also a 401 response should also include some message text, what does that say? In most cases the response will be some JSON like { status: 401, message: `user is not authorized to access or edit space`, spaceId }
3

I added a sleep 1 after every space call and testing that out.

The message I am seeing is {"status":401,"message":"user is not authorized to access or edit space","spaceId":"<space-id>"} (yeah, I expected a 429 too).

1 Like
4

let me know if you get it again after adding sleep

5

I’m still getting the 401’s, but they correspond to all my private spaces. So, I don’t think the KINOPIO_API_KEY environment variable is being expanded properly in my second curl call since it’s a new shell. awk command.

I think I figured how to expand it, rather than using the literal string. Testing that now…

6

Here’s the latest version that works with private spaces:

KINOPIO_API_KEY=<your key> \
curl -H "Authorization: $KINOPIO_API_KEY" https://api.kinopio.club/user/spaces | \
jq -r ".[] | .id + \",\" + (.name | tojson)" | \
awk -v key="$KINOPIO_API_KEY" '{ print "curl -H \"Authorization: "key"\" https://api.kinopio.club/space/"$1" > "$2".json" }' FS=, OFS=, | \
xargs -0 -n1 bash -c
1 Like