S3 Object Storage Client (s3fs)

AstLinux now supports the s3fs (FUSE) client for S3 object storage.

It is required to have access to an S3 Object Storage Server, either locally or in the cloud. Tested providers

When s3fs is active, the S3 object storage will be available to AstLinux at the mountpoint /var/s3fs/mnt

Note: AstLinux 1.5.5 or later is required

Enable s3fs Support:

Select the Network Tab in the web interface.

Edit the user.conf file…

• Define S3FS_URL

## Example for Linode (Akamai) Newark, NJ, defaults to https://s3.amazonaws.com (AWS)
S3FS_URL="https://us-east-1.linodeobjects.com"

• Define S3FS_BUCKET

## Note: optionally append :/PATH to the bucket name to mount at /PATH
S3FS_BUCKET="unique-bucket-name"

Tip -> For additional options, see below: Optional Configuration

• Click “Save Changes”
• Click “Reload/Restart” - [Apply user.conf variables] - _x_ Confirm

Then from the CLI:

service s3fs start

On first start, you will see this error message…

s3fs: The '/mnt/kd/s3fs/passwd/passwd-s3fs' file must exist (ACCESS_KEY_ID:SECRET_ACCESS_KEY), skip start

As suggested, create a file /mnt/kd/s3fs/passwd/passwd-s3fs with a line containing the ACCESS_KEY_ID:SECRET_ACCESS_KEY from your S3 provider.

Then, try again:

service s3fs start

To test if it worked, issue the command mount | grep /var/s3fs/mnt you should see something like:

s3fs on /var/s3fs/mnt type fuse.s3fs (rw,nosuid,nodev,relatime,user_id=0,group_id=0)

The S3 files in the bucket should appear with ls -l /var/s3fs/mnt/

Disable s3fs Support:

First from the CLI:

service s3fs stop

If you don't want s3fs to start on boot:

• Remove or comment-out the S3FS_START variable in user.conf (if defined)

If you don't want to use s3fs anytime soon, best practice is to delete any S3 credentials:

• Remove all files in the /mnt/kd/s3fs/passwd/ directory

The s3fs service does not start by default, but via the CLI, issue service s3fs start to manually mount, and service s3fs stop to manually unmount the S3 bucket.

If you make changes to the s3fs configuration, issue service s3fs restart to apply the changes.

If you want s3fs to start on boot, add to the user.conf file:

S3FS_START="yes"

Refer to editing the user.conf file via the web interface (above).

• Optionally define S3FS_START, useful if starting at boot is desired

## The s3fs service does not start by default, "yes" or "no", defaults to "no"
S3FS_START="yes"

• Optionally define S3FS_PASSWD, useful when switching between multiple providers.

## Name of passwd file in /mnt/kd/s3fs/passwd/, defaults to passwd-s3fs if not defined
S3FS_PASSWD="linode-ro"

• Optionally define S3FS_READONLY, useful when Read/Write access is not desired.

## Mount as Read/Only (ro), "yes" or "no", defaults to "no"
## Note: When "yes" best practice is to have bucket access permissions Read/Only as well
S3FS_READONLY="yes"

• Optionally define S3FS_OPTIONS, necessary with some providers, see Tested providers

## -o options added to s3fs, space separate multiple options
S3FS_OPTIONS="nomultipart"

Tip -> Cloudflare R2 (as of 2024/07), the nomixupload option is needed to write large files.

One interesting use case for s3fs is to provide a Read/Only mount to an S3 bucket containing a custom AstLinux firmware repository.

Most S3 object storage providers allow files to be made “public” and accessed read-only via HTTPS (CORS). The problem is the “world” has access to the firmware. Restricting with a firewall is difficult, or not possible due to scattered devices.

Alternatively, in the same way using S3 object storage, but keep the S3 files “private” and access the firmware directly via s3fs using an access key with read-only privileges to a particular bucket.

For this example, build a custom AstLinux run image and use s3cmd on the build system to upload to a S3 bucket in AstLinux Private Repository format starting with the path custom-firmware-1.x

The s3fs configuration in the user.conf file would be similar to:

S3FS_START="yes"
S3FS_URL="https://us-east-1.linodeobjects.com"
S3FS_BUCKET="unique-bucket-name"
S3FS_PASSWD="linode-ro"
S3FS_READONLY="yes"

At Linode, create a Read/Only “Access Key” limited to the “unique-bucket-name” bucket. Place the ACCESS_KEY_ID:SECRET_ACCESS_KEY credentials in the file: /mnt/kd/s3fs/passwd/linode-ro

In the AstLinux web interface Prefs Tab, define the “Repository URL:” as:

file:///var/s3fs/mnt/custom-firmware-1.x