OpenStack Object Storage (Swift)

The OpenStack Object Store project, known as Swift, offers cloud storage software that allows you to store and retrieve large amounts of data with a simple API. It’s built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bounds.

Swift—like Amazon S3—has an eventual consistency architecture, which makes it ideal for building massive, highly distributed + infrastructures with lots of unstructured data serving global sites.

OpenStack Swift is best suited for backing up and archiving unstructured data, such as documents, images, audio and video files, emails, and virtual machine images.

All objects (data) stored in Swift have a URL

Swift stores 3 replicas of the data by default. This means that total storage / 3 is your billable or usable storage. The number of replicas is configurable on a cluster level.

Swift Architecture – https://docs.openstack.org/swift/latest/overview_architecture.html

Accounts and containers

Each account and container is an individual SQLite database that is distributed across the cluster. An account database contains the list of containers in that account. A container database contains the list of objects in that container.

To keep track of object data locations, each account in the system has a database that references all of its containers, and each container database references each object.

How to use Object Storage (Swift) in different scenarios: –

The Swift Service must first be configured in OpenStack. Once Swift has been set up, you can view the Object storage area of the OpenStack Dashboard.

I have Configured Swift Service.

If you want to learn about different services in Swift, refer to the Swift recapture doc link above.

Login to the Openstack Dashboard

You can see the object store section in the dashboard.

Scenario 1:

Upload a sample PDF file and grant public access to it: –

Object store > Container > click on Container.

Enter the Container Name, select storage policy (You can create different policies based on requirements; to understand the storage policy, refer to the above swift architecture doc link), and select Container Access as Public.

The container was created successfully. You see the public access URL. (If you wish to turn the container from public to private, just uncheck the Public Access option.)

Click the upload button to add the file to the container. Browse the local file and select the Upload File option.

File Uploaded Successfully.

Share the file with the whole public users. Right-click on the Public Access Link and select Open Link in New Tab.

It will display all of the file information within the container. If you wish to download a specific file within the container. Just add the file name at the end of the link.

Added file name at end of the link (/ImageGuide.pdf). Press enter to download the file)

The file is available to view and download. The link can be shared. Anyone can download the file by clicking on the URL provided below.

If you wish to make the Public file private. You may just uncheck the public access option.

Public Access disabled to the container.

Once Public access is disabled, the container cannot be accessed via the link.

Secnario2:

For example, ABC has different finance and marketing teams.

Marketing Team members – maruser0

Finance Team members – finuser0

The finance team has two different containers.

1. Internal
   
2. customer
   

Fuser0 is a member of the Finance team. Muser0 wants to provide him read access to the internal container and write access to the customer container.

Actions to complete the scenario mentioned above are as follows: –

Create two projects in Openstack – Marketing and Finance.

Create a muser0 user and assign a member role in a marketing project.

Create the fuser0 user and assign a member role in a Finance project.

Login using muser0 and create a container Internal and customer and upload the file

Assigning access permission to the containers

Creating the projects (I’m using CLI, but you can also use the dashboard to create projects and users)

For CLI access, Install client tools. I am using CentOS 9 (https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html)

pip3 install python-openstackclient

pip3 install python-swiftclient

Download the admin openrc file from the dashboard.

After downloading the openrc file. Run the following command

$ source admin-openrc.sh

Enter admin password

# Project creation

$openstack project create –description “project for the Marketing Team” marketing

$openstack project create –description “project for the Finance Team” finance

# User Creation

$openstack user create –project marketing –password Test@2024 muser0

$openstack user create –project finance –password Test@2024 fuser0

# Assiging the Role

$openstack role add –user muser0 –project marketing member

$openstack role add –user fuser0 –project finance member

Project, User and role assignments completed successfully.

Crating the container and uploading the sample files

Download marketing and finance openrc files.

Switch to the marketing project to create a container and upload example files.

# Creating the containers and uploading sample files

$swift upload internal ImageGuide.pdf

$swift upload customer ImageGuide1.pdf

Assign access permission to the fuser0 (Finance Team member)

$swift post internal -r “finance:fuser0”

Internal container Read access to the fuser0: The user can download but not upload or remove existing files in the internal container.

$swift post customer -r :finance:fuser0”

$swift post customer -w “finance:fuser0”

Customer container Write access to the fuser0: The user can download, upload or remove existing files in the customer container.

To access the marketing files, you’ll need a marketing API connection. To retrieve the marketing API link, use the following command.

$swift auth

Copy OS_STORAGE_URL

Switch to the finance project to validate access to the containers.

$ source finance-openrc.sh

Customer container has read and write access.You can upload new files to the container and remove old files.

$ swift –os-storage-url=http://172.16.24.90:8080/v1/AUTH_8a2ae840f9b2479ca49cd7ed356e9968 list customer

$ swift –os-storage-url=http://172.16.24.90:8080/v1/AUTH_8a2ae840f9b2479ca49cd7ed356e9968 upload customer ImageGuide3.pdf

$ swift –os-storage-url=http://172.16.24.90:8080/v1/AUTH_8a2ae840f9b2479ca49cd7ed356e9968 list customer

For deleting the existing file

$ swift –os-storage-url=http://172.16.24.90:8080/v1/AUTH_8a2ae840f9b2479ca49cd7ed356e9968 delete customer ImageGuide3.pdf

$ swift –os-storage-url=http://172.16.24.90:8080/v1/AUTH_8a2ae840f9b2479ca49cd7ed356e9968 list customer

You may schedule the deletion of files.

$ swift post customer ImageGuide.pdf -H “X-Delete-After:15”

File delete after 15 sec.

$ Swift post internal ImageGuide -H “X-Delete-At:1716121930”

Using online tools, you may convert date and time to Unix timestamps.

1716121930 – Sunday, May 19, 2024 12:32:10 PM

The file will be deleted on Sunday, May 19, 2024, 12:32:10 PM