Open file limits with Scalr scripts

Ross Kilgariff's Avatar

Ross Kilgariff

24 Sep, 2015 12:39 PM

Hi,

One of our services started having problems at around 8AM UTC.

The problem is that our service exceeds the max file descriptors limit, even though the system's limit settings are very high.

On further inspection, it seems that any scripts run through Scalr inherit a file descriptor limit of 1024, while logging in using SSH and running 'ulimit -Sn' shows the correct system setting.

Nothing was changed in our server configuration at this time - was this caused by a change to Scalarizr?

I am currently working around this problem by calling 'ulimit' to set the limits manually before starting the service. Is there a better way to ensure Scalarizr uses the correct limits when running scripts?

Thanks,
Ross

  1. Support Staff 1 Posted by Marat Komarov on 25 Sep, 2015 08:40 AM

    Marat Komarov's Avatar

    Scalarizr doesn't have a special ulimit setting. Most likely OS wide ulimit is updated system after Scalarizr startup.

    Also take in mind that Scalarizr couldn't be restarted during server bootstrap: this will stop initialization. You can only restart it on HostUp event.

    Regards,
    Marat

  2. 2 Posted by Ross Kilgariff on 25 Sep, 2015 09:32 AM

    Ross Kilgariff's Avatar

    Hi Marat,

    Thanks for your reply.

    The OS-wide limit for the server was _not_ changed. Using SSH to connect to the server and running 'ulimit -Sn' shows 81920, but putting 'ulimit -Sn' in a Scalr script and running it manually shows '1024'.

    The file /etc/security/limits.conf has limits set up for all users as follows:

    * hard nofile 81920
    * soft nofile 81920
    root hard nofile 81920
    root soft nofile 81920

    No changes were made by us when this issue occurred - our service simply stopped working. This is why I mention that something in Scalarizr may have changed.

    We have a workaround now, but mentioning in case it affects other customers.

    Thanks,
    Ross

  3. marc closed this discussion on 22 Jul, 2016 08:32 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac