Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-580

pegasus-analyzer has problems querying MySQL stampede db

XMLWordPrintable

      pegasus-analyzer $PWD
      pegasus-analyzer: initializing...

      ***********************************Summary************************************

      Total jobs : 13 (100.00%)

      1. jobs succeeded : 5 (38.46%)
      2. jobs failed : 3 (23.08%)
      3. jobs unsubmitted : 5 (38.46%)

      *****************************Failed jobs' details*****************************
      Traceback (most recent call last):
      File "/opt/pegasus/default/bin/pegasus-analyzer", line 1301, in ?
      analyze_db(options.config_properties)
      File "/opt/pegasus/default/bin/pegasus-analyzer", line 919, in analyze_db
      my_info = workflow_stats.get_job_instance_info(my_job[0])[0]
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/python/netlogger/analysis/workflow/stampede_statistics.py", line 921, in get_job_instance_info
      return q.all()
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/orm/query.py", line 1492, in all
      return list(self)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/orm/query.py", line 1604, in _iter_
      return self._execute_and_instances(context)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/orm/query.py", line 1609, in _execute_and_instances
      mapper=self._mapper_zero_or_none())
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/orm/session.py", line 700, in execute
      return self._connection_for_bind(engine, close_with_result=True).execute(
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/base.py", line 1194, in execute
      params)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/base.py", line 1274, in _execute_clauseelement
      return self.__execute_context(context)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/base.py", line 1305, in __execute_context
      context.parameters[0], context=context)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/base.py", line 1404, in _cursor_execute
      context)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/base.py", line 1397, in _cursor_execute
      context)
      File "/opt/pegasus/pegasus-4.0.0/lib/pegasus/externals/python/sqlalchemy/engine/default.py", line 299, in do_execute
      cursor.execute(statement, parameters)
      File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 163, in execute
      self.errorhandler(self, exc, value)
      File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 35, in defaulterrorhandler
      raise errorclass, errorvalue
      sqlalchemy.exc.OperationalError: (OperationalError) (1054, "Unknown column 'job_instance.job_instance_id' in 'where clause'") 'SELECT job_instance.job_instance_id AS job_instance_job_instance_id, job_instance.site AS job_instance_site, job_instance.stdout_file AS job_instance_stdout_file, job_instance.stderr_file AS job_instance_stderr_file, job_instance.stdout_text AS job_instance_stdout_text, job_instance.stderr_text AS job_instance_stderr_text, job_instance.work_dir AS job_instance_work_dir, (SELECT workflow.submit_dir \nFROM workflow \nWHERE workflow.wf_id = job_instance.subwf_id) AS subwf_dir, (SELECT job.exec_job_id \nFROM job \nWHERE job.job_id = job_instance.job_id) AS job_name, (SELECT job.submit_file \nFROM job \nWHERE job.job_id = job_instance.job_id) AS submit_file, (SELECT job.executable \nFROM job \nWHERE job.job_id = job_instance.job_id) AS executable, (SELECT job.argv \nFROM job \nWHERE job.job_id = job_instance.job_id) AS argv, (SELECT workflow.submit_dir \nFROM workflow \nWHERE workflow.wf_id = %s) AS submit_dir, (SELECT jobstate.state \nFROM jobstate, (SELECT max(jobstate.jobstate_submit_seq) AS max_job_submit_seq \nFROM jobstate \nWHERE jobstate.job_instance_id = job_instance.job_instance_id) AS anon_1 \nWHERE jobstate.job_instance_id = job_instance.job_instance_id AND jobstate.jobstate_submit_seq = anon_1.max_job_submit_seq) AS state, (SELECT invocation.executable \nFROM invocation \nWHERE invocation.job_instance_id = job_instance.job_instance_id AND invocation.task_submit_seq = %s) AS pre_executable, (SELECT invocation.argv \nFROM invocation \nWHERE invocation.job_instance_id = job_instance.job_instance_id AND invocation.task_submit_seq = %s) AS pre_argv, (SELECT host.hostname \nFROM host \nWHERE host.host_id = job_instance.host_id) AS hostname \nFROM job_instance \nWHERE job_instance.job_instance_id = %s' (2L, -1, -1, 19L)

      However, the column does exist in the database:

      mysql> describe job_instance;
      ------------------------------------------------------------+

      Field Type Null Key Default Extra

      ------------------------------------------------------------+

      job_instance_id bigint(20) NO PRI NULL auto_increment
      job_id bigint(20) NO MUL NULL  
      host_id bigint(20) YES MUL NULL  
      job_submit_seq int(11) NO   NULL  
      sched_id varchar(255) YES   NULL  
      site varchar(255) YES   NULL  
      user varchar(255) YES   NULL  
      work_dir text YES   NULL  
      cluster_start decimal(16,6) YES   NULL  
      cluster_duration decimal(10,3) YES   NULL  
      local_duration decimal(10,3) YES   NULL  
      subwf_id bigint(20) YES MUL NULL  
      stdout_file varchar(255) YES   NULL  
      stdout_text text YES   NULL  
      stderr_file varchar(255) YES   NULL  
      stderr_text text YES   NULL  
      stdin_file varchar(255) YES   NULL  
      multiplier_factor int(11) NO   NULL  
      exitcode int(11) YES   NULL  

      ------------------------------------------------------------+
      19 rows in set (0.00 sec)

      I'll attach the submit dir next.

            Assignee:
            mmgoode Monte Goode
            Reporter:
            voeckler Jens Voeckler
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: