radosgw – rados REST gateway

Synopsis

radosgw

Description

radosgw is an HTTP REST gateway for the RADOS object store, a partof the Ceph distributed storage system. It is implemented as a FastCGImodule using libfcgi, and can be used in conjunction with any FastCGIcapable web server.

Options

  • -c ceph.conf, —conf=ceph.conf
  • Use ceph.conf configuration file instead of the default/etc/ceph/ceph.conf to determine monitor addresses during startup.
  • -m monaddress[:port]
  • Connect to specified monitor (instead of looking through ceph.conf).
  • -i ID, —id ID
  • Set the ID portion of name for radosgw
  • -n TYPE.ID, —name TYPE.ID
  • Set the rados user name for the gateway (eg. client.radosgw.gateway)
  • —cluster NAME
  • Set the cluster name (default: ceph)
  • -d
  • Run in foreground, log to stderr
  • -f
  • Run in foreground, log to usual location
  • —rgw-socket-path=path
  • Specify a unix domain socket path.
  • —rgw-region=region
  • The region where radosgw runs
  • —rgw-zone=zone
  • The zone where radosgw runs

Configuration

Earlier RADOS Gateway had to be configured with Apache and mod_fastcgi.Now, mod_proxy_fcgi module is used instead of mod_fastcgi.mod_proxy_fcgi works differently than a traditional FastCGI module. Thismodule requires the service of mod_proxy which provides support for theFastCGI protocol. So, to be able to handle FastCGI protocol, both mod_proxyand mod_proxy_fcgi have to be present in the server. Unlike mod_fastcgi,mod_proxy_fcgi cannot start the application process. Some platforms havefcgistarter for that purpose. However, external launching of applicationor process management may be available in the FastCGI application frameworkin use.

Apache can be configured in a way that enables mod_proxy_fcgi to be usedwith localhost tcp or through unix domain socket. mod_proxy_fcgi that doesn’tsupport unix domain socket such as the ones in Apache 2.2 and earlier versions ofApache 2.4, needs to be configured for use with localhost tcp. Later versions ofApache like Apache 2.4.9 or later support unix domain socket and as such theyallow for the configuration with unix domain socket instead of localhost tcp.

The following steps show the configuration in Ceph’s configuration file i.e,/etc/ceph/ceph.conf and the gateway configuration file i.e,/etc/httpd/conf.d/rgw.conf (RPM-based distros) or/etc/apache2/conf-available/rgw.conf (Debian-based distros) with localhosttcp and through unix domain socket:

  • For distros with Apache 2.2 and early versions of Apache 2.4 that uselocalhost TCP and do not support Unix Domain Socket, append the followingcontents to /etc/ceph/ceph.conf:
  1. [client.radosgw.gateway]
  2. host = {hostname}
  3. keyring = /etc/ceph/ceph.client.radosgw.keyring
  4. rgw socket path = ""
  5. log file = /var/log/ceph/client.radosgw.gateway.log
  6. rgw frontends = fastcgi socket_port=9000 socket_host=0.0.0.0
  7. rgw print continue = false
  • Add the following content in the gateway configuration file:

For Debian/Ubuntu add in /etc/apache2/conf-available/rgw.conf:

  1. <VirtualHost *:80>
  2. ServerName localhost
  3. DocumentRoot /var/www/html
  4.  
  5. ErrorLog /var/log/apache2/rgw_error.log
  6. CustomLog /var/log/apache2/rgw_access.log combined
  7.  
  8. # LogLevel debug
  9.  
  10. RewriteEngine On
  11.  
  12. RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
  13.  
  14. SetEnv proxy-nokeepalive 1
  15.  
  16. ProxyPass / fcgi://localhost:9000/
  17.  
  18. </VirtualHost>

For CentOS/RHEL add in /etc/httpd/conf.d/rgw.conf:

  1. <VirtualHost *:80>
  2. ServerName localhost
  3. DocumentRoot /var/www/html
  4.  
  5. ErrorLog /var/log/httpd/rgw_error.log
  6. CustomLog /var/log/httpd/rgw_access.log combined
  7.  
  8. # LogLevel debug
  9.  
  10. RewriteEngine On
  11.  
  12. RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
  13.  
  14. SetEnv proxy-nokeepalive 1
  15.  
  16. ProxyPass / fcgi://localhost:9000/
  17.  
  18. </VirtualHost>
  • For distros with Apache 2.4.9 or later that support Unix Domain Socket,append the following configuration to /etc/ceph/ceph.conf:
  1. [client.radosgw.gateway]
  2. host = {hostname}
  3. keyring = /etc/ceph/ceph.client.radosgw.keyring
  4. rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
  5. log file = /var/log/ceph/client.radosgw.gateway.log
  6. rgw print continue = false
  • Add the following content in the gateway configuration file:

For CentOS/RHEL add in /etc/httpd/conf.d/rgw.conf:

  1. <VirtualHost *:80>
  2. ServerName localhost
  3. DocumentRoot /var/www/html
  4.  
  5. ErrorLog /var/log/httpd/rgw_error.log
  6. CustomLog /var/log/httpd/rgw_access.log combined
  7.  
  8. # LogLevel debug
  9.  
  10. RewriteEngine On
  11.  
  12. RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
  13.  
  14. SetEnv proxy-nokeepalive 1
  15.  
  16. ProxyPass / unix:///var/run/ceph/ceph.radosgw.gateway.fastcgi.sock|fcgi://localhost:9000/
  17.  
  18. </VirtualHost>

Please note, Apache 2.4.7 does not have Unix Domain Socket support init and as such it has to be configured with localhost tcp. The Unix DomainSocket support is available in Apache 2.4.9 and later versions.

  • Generate a key for radosgw to use for authentication with the cluster.
  1. ceph-authtool -C -n client.radosgw.gateway --gen-key /etc/ceph/keyring.radosgw.gateway
  2. ceph-authtool -n client.radosgw.gateway --cap mon 'allow rw' --cap osd 'allow rwx' /etc/ceph/keyring.radosgw.gateway
  • Add the key to the auth entries.
  1. ceph auth add client.radosgw.gateway --in-file=keyring.radosgw.gateway
  • Start Apache and radosgw.

Debian/Ubuntu:

  1. sudo /etc/init.d/apache2 start
  2. sudo /etc/init.d/radosgw start

CentOS/RHEL:

  1. sudo apachectl start
  2. sudo /etc/init.d/ceph-radosgw start

Usage Logging

radosgw maintains an asynchronous usage log. It accumulatesstatistics about user operations and flushes it periodically. Thelogs can be accessed and managed through radosgw-admin.

The information that is being logged contains total data transfer,total operations, and total successful operations. The data is beingaccounted in an hourly resolution under the bucket owner, unless theoperation was done on the service (e.g., when listing a bucket) inwhich case it is accounted under the operating user.

Following is an example configuration:

  1. [client.radosgw.gateway]
  2.     rgw enable usage log = true
  3.     rgw usage log tick interval = 30
  4.     rgw usage log flush threshold = 1024
  5.     rgw usage max shards = 32
  6.     rgw usage max user shards = 1

The total number of shards determines how many total objects hold theusage log information. The per-user number of shards specify how manyobjects hold usage information for a single user. The tick intervalconfigures the number of seconds between log flushes, and the flushthreshold specify how many entries can be kept before resorting tosynchronous flush.

Availability

radosgw is part of Ceph, a massively scalable, open-source, distributedstorage system. Please refer to the Ceph documentation at http://ceph.com/docs formore information.

See also

ceph(8)radosgw-admin(8)