-
Type: Bug
-
Resolution: Fixed
-
Priority: Major
-
Affects Version/s: master, 3.1
-
Component/s: statistics visualization and debugging tools
-
None
While running pegasus plots on a synthetic scec run I see eventually an operational error saying too many connections()
The SCEC run has around 77 sub workflows. The plotting code for each sub workflow has to initialize an object of the Stampede API , that opens a connection in SQLAlchemy. I think in this case, the connections are not being closed automatically.
Probably an explicit close connection method should be provided?
2011-08-05 15:17:23,154:pegasus-plots:create_charts:356: INFO: Generating graphs/charts for the workflow 34a04ade-26c3-440f-8cdf-d3e39d39abc0 ...
.....
2011-08-05 15:17:25,455:pegasus-plots:create_charts:356: INFO: Generating graphs/charts for the workflow 86139274-e17d-4328-b941-4547d13d6630 ...
2011-08-05 15:17:26,457:pegasus-plots:create_charts:356: INFO: Generating graphs/charts for the workflow 6756b543-7641-45a3-9889-a0639133824b ...
2011-08-05 15:17:26,638:populate.py:get_wf_stats:354: WARNING: Traceback (most recent call last):
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/Pegasus/plots_stats/plots/populate.py", line 350, in get_wf_stats
workflow_stampede_stats = StampedeStatistics(global_db_url , expand)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/netlogger/analysis/workflow/stampede_statistics.py", line 164, in _init_
SQLAlchemyInit._init_(self, connString, initializeToPegasusDB)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/netlogger/analysis/modules/base.py", line 129, in __init_
initFunction(self.db, self.metadata, kw=dialect_kw)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/netlogger/analysis/schema/stampede_schema.py", line 408, in initializeToPegasusDB
metadata.create_all(db)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/schema.py", line 2119, in create_all
bind.create(self, checkfirst=checkfirst, tables=tables)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/engine/base.py", line 1701, in create
connection=connection, **kwargs)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/engine/base.py", line 1738, in _run_visitor
conn = self.contextual_connect(close_with_result=False)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/engine/base.py", line 1832, in contextual_connect
self.pool.connect(),
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 183, in connect
return _ConnectionFairy(self).checkout()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 370, in _init_
rec = self._connection_record = pool.get()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 214, in get
return self.do_get()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 733, in do_get
con = self.create_connection()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 148, in create_connection
return _ConnectionRecord(self)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 254, in _init_
self.connection = self.__connect()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/pool.py", line 320, in __connect
connection = self.__pool._creator()
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/engine/strategies.py", line 76, in connect
return dialect.connect(*cargs, **cparams)
File "/data/scratch/vahi/software/install/pegasus/default/lib/python/sqlalchemy/engine/default.py", line 249, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-x86_64.egg/MySQLdb/_init_.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-x86_64.egg/MySQLdb/connections.py", line 187, in _init_
super(Connection, self)._init_(*args, **kwargs2)
OperationalError: (OperationalError) (1040, 'Too many connections') None None
vahi@obelix:/data/scratch/vahi/examples/synthetic-scec/dags/vahi/pegasus/Stampede-Test$ logger.warning(traceback