GCS Limit Collection HTTP Upload Size

Globus Connect Server utilizes the Apache HTTP web server to provide access to collections. By default, there may not be a limit to the size of an HTTP PUT request (upload) that Apache will accept, which can lead to resource exhaustion if users attempt to upload very large files. This document describes how to set a maximum upload size for collections on a GCS endpoint by modifying the Apache HTTP configuration.

1. Limiting Upload Size with Apache LimitRequestBody

The Apache HTTP LimitRequestBody directive can be used to specify the maximum allowed size of an HTTP file upload. By placing this directive in the appropriate configuration file, administrators can restrict the size of uploads to collections on a GCS endpoint. The size is specified in bytes, so for example, to limit uploads to 100 MB, you would set LimitRequestBody to 104857600. When a user attempts to upload a file that exceeds this limit, they will receive a 413 Request Entity Too Large error code error response. A value of 0 sets the maximum upload file size to unlimited.

The default Apache configuration for GCS collections does not include a LimitRequestBody directive, which means the maximum upload size depends on any site-specific HTTPD configurations or the default settings of the Apache installation. In Apache HTTP Server 2.4.53 and earlier, the default value was 0 (unlimited). In Apache HTTP Server 2.4.54 and later, the default value is 2147483647 bytes (approximately 2 GB). Some Linux distributions may include a default LimitRequestBody directive in their Apache configuration files or even distribute Apache with a custom compiled default upload limit which could differ from the standard Apache defaults.

2. Configuration File Locations

As of GCS 5.4.93, administrators may Extend Collection Virtual Host Configurations to customize the Apache HTTP configuration for collections on a GCS endpoint. Using this feature, administrators can place a LimitRequestBody directive in the appropriate configuration file to set a maximum upload size for collections.

Note
It is the administrator’s responsibility to place these vhost configurations on all nodes in the endpoint. GCS does not manage, synchronize, or replicate these manual configuration files between nodes in a multi-node deployment.

In this example, we will set a maximum upload size of 100 MB for the collection with the ID a76e9c7d-b1cc-430e-8ac2-733aa4daecf3. To do this, we will create a configuration file at /etc/globus/httpd/a76e9c7d-b1cc-430e-8ac2-733aa4daecf3.conf with the following content:

LimitRequestBody 104857600

Restart the Apache HTTP service for the changes to take effect.

Ubuntu, Debian, SUSE
$ sudo systemctl restart apache2.service
Fedora, CentOS, RedHat
$ sudo systemctl restart httpd.service

3. Verifying Custom Upload Limits

In order to verify that the custom upload limit is functioning correctly, use the Globus Webapp to attempt to upload a file to the collection that exceeds the specified limit. You should receive a 413 Request Entity Too Large error response, indicating that the upload was rejected due to the size limit.

Visit the collection in the Globus Webapp by searching for it by name or ID using the collection search feature at https://app.globus.org/collections. Once you have located the collection, attempt to upload a file larger than the specified limit (in this case, larger than 100 MB) using the Webapp’s "Upload" feature.

The upload will begin, but once it hits the configured upload limit, it will be rejected and you will see an error in the lower left-hand corner of the Webapp with the message Request Entity Too Large.

See Verifying Changes in the Extend Collection Virtual Host Configurations guide for more details on how to verify correct functionality after applying custom Apache configurations.