Uploaded image for project: 'Pegasus'

glite submission results in quoted environment variable keys

XMLWordPrintable

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major Major
    • master, 4.5.0, 4.4.1
    • Affects Version/s: master, 4.4.0
    • Component/s: Pegasus Planner
    • None
    • Environment:
      Glite submissions to local pbs cluster via condor

      looks like the way condor and blahp sets the environment variables for the jobs using +remote_ce_requirements is broken.

      condor-admin ticket 27552

      In Pegasus, we use +remote_cerequirements to achieve this.

      My test condor job is like this
      ######################################################################
      stream_error = false
      stream_output = false
      environment = PEGASUS_HOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs;
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE 0,KEY1=VALUE1\""
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE0,KEY1=VALUE1\""
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1"
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=\"VALUE 0\",KEY1=VALUE1"
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1\""

      copy_to_spool = false
      error = test.err
      executable = /bin/env
      grid_resource = pbs
      log = test.log
      notification = NEVER
      output = test.out

      transfer_executable = false
      universe = grid

      queue

      Here are my experiments, and none seem to work correctly, especially if the value has a white space.

      If the env variable has a whitespace in value
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE 0,KEY1=VALUE1\""
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out

      ————————————————————————————————————————————————————————————————————————————
      No whitespace in value
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE0,KEY1=VALUE1\""
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      "PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      KEY1=VALUE1"
      KEY0=VALUE0

      The first env variable name has “ and last env value has “ in the end

      ————————————————————————————————————————————————————————————————————————————
      Values are single quoted
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1"
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out

      Nothing was set
      ————————————————————————————————————————————————————————————————————————————
      Values are escaped double quotes
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=\"VALUE 0\",KEY1=VALUE1"

      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      [vahi@hpc-jak condor-glite]$

      ————————————————————————————————————————————————————————————————————————————
      MYENV is within quotes, and env variable values are in single quotes
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1\""

      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      ————————————————————————————————————————————————————————————————————————————

            Assignee:
            mayani Rajiv Mayani
            Reporter:
            vahi Karan Vahi
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: