Unable to start db server

Patrick McAndrew's Avatar

Patrick McAndrew

22 Jul, 2015 10:58 AM

2015-07-22 10:44:01,123+00:00 - WARNING - scalarizr.messaging.p2p.producer - Caught exception
2015-07-22 10:44:43,619+00:00 - ERROR - scalarizr.ops.system.init - Operation "system.init" (id: f68b2233-5036-43b8-a0cd-bc468820521f) failed. Reason:
Traceback (most recent call last):
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/api/operation.py", line 191, in _in_progress
    self._completed(self.func(self, *self.func_args, **self.func_kwds))
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/handlers/lifecycle.py", line 476, in handler
    self.send_message(Messages.BEFORE_HOST_UP, broadcast=True, handle_before_host_up=True)
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/handlers/__init__.py", line 66, in send_message
    return cons.handle_before_host_up(msg)
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/messaging/p2p/consumer.py", line 288, in message_handler
    self._handle_one_message(message, queue, store)
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/messaging/p2p/consumer.py", line 232, in _handle_one_message
    store.mark_as_handled(message.id)
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/messaging/p2p/__init__.py", line 209, in mark_as_handled
    cur.execute(sql, [1, msg_s, message_id, 1])
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/util/sqlite_server.py", line 69, in execute
    self._execute_result = self._call('cursor_execute', args)
  File "/opt/scalarizr/embedded/lib/python2.7/site-packages/scalarizr-3.6.7-py2.7.egg/scalarizr/util/sqlite_server.py", line 45, in _call
    raise TimeoutError()
TimeoutError

  1. 1 Posted by Patrick McAndre... on 22 Jul, 2015 11:00 AM

    Patrick McAndrew's Avatar

    vaguely related to http://support.scalr.net/discussions/problems/25442-regression-wroot-percona-password-unable-to-start-bundled-db-image as think some type of manual patch was applied to the server :P

    however, this time server is a t1.small

    farm: 11159

  2. 2 Posted by Patrick McAndre... on 22 Jul, 2015 11:02 AM

    Patrick McAndrew's Avatar

    would be nice if the underlying issue can be resolved once and for all, as having to go through support each time we want to build a db server is incredibly inefficient

  3. 3 Posted by Patrick McAndre... on 22 Jul, 2015 11:34 AM

    Patrick McAndrew's Avatar

    ok, I think the issue is due to us running dd to create a swap file at the beginning of the beforehostup process.

    Shouldn't the scalr processes that are timeout dependent should be allowed to complete prior to running any user processes.

  4. Support Staff 4 Posted by Marat Komarov on 22 Jul, 2015 11:44 AM

    Marat Komarov's Avatar

    TimeoutError you're experiencing is an SQL query timeout to a local SQLite database. It coudn't be removed in any way, but we could find and understand why your script caused such behavior.

    Could you please provide script ID you've removed?

  5. 5 Posted by Patrick McAndre... on 22 Jul, 2015 11:48 AM

    Patrick McAndrew's Avatar

    The timeout is occurring in an initialization process, correct? My point is that the sqlite db should be initialized first, then start to run the user scripts.

    Script id: 8323 - I just reduce the size of the swapfile to get it to run.

  6. Support Staff 6 Posted by Marat Komarov on 22 Jul, 2015 12:06 PM

    Marat Komarov's Avatar

    No, it doesn't work the way you're thinking. Thanks for a script id.

    Regards,
    Marat

  7. Support Staff 7 Posted by Marat Komarov on 22 Jul, 2015 12:08 PM

    Marat Komarov's Avatar

    What was the previous GV_SWAPSIZE_PERCENT value?

  8. 8 Posted by Patrick McAndre... on 22 Jul, 2015 12:47 PM

    Patrick McAndrew's Avatar

    150

  9. 9 Posted by Patrick McAndre... on 25 Sep, 2015 09:13 AM

    Patrick McAndrew's Avatar

    Still is still a periodic issue. What's the eta on a fix?

  10. Support Staff 10 Posted by Marat Komarov on 25 Sep, 2015 09:42 AM

    Marat Komarov's Avatar

    No obvious bug in Scalarizr that we need to fix. It's a kind of issue that depends from environment and scripts.

  11. 11 Posted by Patrick McAndre... on 25 Sep, 2015 09:46 AM

    Patrick McAndrew's Avatar

    How is it not a bug with scalr?

    Creating a swapfile is a standard thing to do and works on all of our server images except the db server.

  12. Support Staff 12 Posted by Marat Komarov on 25 Sep, 2015 09:53 AM

    Marat Komarov's Avatar

    All other servers are also managed by Scalr? So maybe the problem is with db server image?

  13. 13 Posted by Patrick McAndre... on 25 Sep, 2015 10:01 AM

    Patrick McAndrew's Avatar

    If I remove the create swapfile script, it works fine. If the swapfile script is present, then the server fails to boot. The failure occurs inside one of the scalr processes. The swapfile script works fine on other servers, so its not an issue the swapfile script itself.

  14. marc closed this discussion on 22 Jul, 2016 08:33 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac