Adventures of a Small Time OpenStack Sysadmin Chapter 042 - Swift Object Storage Service

Adventures of a Small Time OpenStack Sysadmin relate the experience of converting a small VMware cluster into two small OpenStack clusters, and the adventures and friends I made along the way.

Adventures of a Small Time OpenStack Sysadmin Chapter 042 - Swift Object Storage Service

First, links to some reference docs I used:

Swift Docs Page

https://docs.openstack.org/swift/yoga/

Kolla-Ansible Deployment Configuration Reference for Swift

https://docs.openstack.org/kolla-ansible/yoga/reference/storage/swift-guide.html

Swift Ops Runbook

https://docs.openstack.org/swift/yoga/ops_runbook/index.html

Configure Swift

Configuring Swift is a lot of work compared to other services.  It's pretty rough.

There's a bug in the docs, and/or kolla-ansible, where if you follow the installation instructions, storage policy index 0 is not completely configured, AND there's an auth issue where you have to set a delayed setting if you want to access storage policies via the API.  So that means you CAN use the CLI to create containers but Horizon web UI can NOT create containers until those two settings are fixed.  I opened a bug report at Kolla-Ansible explaining what the bug is, and how to fix it.

I found another problem relating to "swift_delay_auth_decision".  I would strongly suggest that it is a bug for Kolla-Ansible to permit a user to configure "cinder_backup_driver: swift" at the same time as the default configuration of "swift_delay_auth_decision: no" because that'll crash the cinder_backup container as soon as a volume backup is attempted. At least as of Yoga.

Maybe I should not have opened three separate problems in the same bug report, but since no one is working on those bug reports a month later it probably doesn't matter:

https://bugs.launchpad.net/kolla-ansible/+bug/1980880

Next, the rings. I have to set up Swift rings by hand as per:

https://docs.openstack.org/kolla-ansible/yoga/reference/storage/swift-guide.html

Note that the kolla-ansible docs provide an opaque process using docker to generate rings, with a pointer to the swift docs as an explanation, whereas the swift docs use a completely different method. So that's confusing.

See the openstack-scripts/ringmaker scripts at:

https://gitlab.com/SpringCitySolutionsLLC/openstack-scripts/-/tree/master/ringmaker

As for plain old Kolla-Ansible configuration, its pretty simple.  The /etc/kolla/globals.d/swift.yml has a link:

https://gitlab.com/SpringCitySolutionsLLC/openstack-scripts/-/blob/master/backups/os6.cedar.mulhollon.com/globals.d/swift.yml

And it looks like this:

# /etc/kolla/globals.d/swift.yml
#
# On OS6 for cluster 2
#
---

enable_swift: "yes"
enable_swift_recon: "yes"
swift_delay_auth_decision: "yes"
# Later on...
# swift_storage_interface: "bond34.30"
# swift_replication_interface: "{{ swift_storage_interface }}"

#

The "Later on" is for eventually moving swift traffic off the management VLAN onto a dedicated pre-existing storage VLAN.  There would be zero performance gain and I'm not using Swift much so the motivation level has been low on that one, but "eventually" I will push Storage and Overlay onto their dedicated networks toward the end of the Plan 3.0 era or maybe Plan 4.0 timeframe.

Install CLI

See the script installcli.sh:

https://gitlab.com/SpringCitySolutionsLLC/openstack-scripts/-/blob/master/installcli/installcli.sh

Or just run:

pip install python-swiftclient -c https://releases.openstack.org/constraints/upper/yoga

Volume Backup

https://docs.openstack.org/cinder/yoga/admin/volume-backups.html

https://docs.openstack.org/cinder/yoga/admin/volume-backups-export-import.html

I have an imitation of a backup system working now with Cinder and Swift.  However, note that I can spin up and ansible configure a new instance in less than ten minutes whereas a backup, and presumably a restore, could take over a half hour during my experiments, and I already have a disciplined system of not storing data on servers unless its on an otherwise backed-up NFS server.  So I'm not thinking I'll be doing very much OpenStack level backup.  If orchestration and automation is faster than restoring a backup...

rclone

https://rclone.org/swift/

Install and use rclone software, which is kind of a "rsync for clouds" that is Swift compatible, which could be incredibly useful for some applications.

apt-get install rclone

This is a big job, for another day.  Presently I'm not using Swift other than to store Glance images, so I have little to copy right now...

Demonstration Scripts

These can be found at:

https://gitlab.com/SpringCitySolutionsLLC/openstack-scripts/-/tree/master/demos/swift

./create.sh

This creates a temp file containing a timestamp, creates a container named demo, uploads the temp file as an object into that container, and deletes the local copy of the temp file.

./show.sh

This script shows the demo container's status, lists the contents of the demo container, and dumps the tempfile object to stdout so you can see the contents of the file (which should be a simple one line timestamp)

./delete.sh

This merely deletes the demo container and every object inside it.

The good news, is after some pain installing it, Swift works really well.  The bad news is I currently am not using it other than as a backend for glance.  I have some future plans to stash snapshot style backups into Swift and there are some cool utilities that gateway Swift into a filesystem.  It would make a truly awful transactional database backend, but its a pretty cool way to store relatively permanent data like home movies.

Tomorrow, some new (to me) services, Zun and Kuryr.  I'm pretty excited because manual installation of those two seems nearly insurmountable, but Kolla-Ansible supposedly makes it easy.

Stay tuned for the next chapter!