Uploaded image for project: 'Pegasus'
  1. Pegasus
  2. PM-802

glite submission results in quoted environment variable keys

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master, 4.4.0
    • Fix Version/s: master, 4.5.0, 4.4.1
    • Component/s: Pegasus Planner
    • Labels:
      None
    • Environment:
      Glite submissions to local pbs cluster via condor

      Description

      looks like the way condor and blahp sets the environment variables for the jobs using +remote_ce_requirements is broken.

      condor-admin ticket 27552

      In Pegasus, we use +remote_cerequirements to achieve this.


      My test condor job is like this
      ######################################################################
      stream_error = false
      stream_output = false
      environment = PEGASUS_HOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs;
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE 0,KEY1=VALUE1\""
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE0,KEY1=VALUE1\""
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1"
      #+remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=\"VALUE 0\",KEY1=VALUE1"
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1\""

      copy_to_spool = false
      error = test.err
      executable = /bin/env
      grid_resource = pbs
      log = test.log
      notification = NEVER
      output = test.out

      transfer_executable = false
      universe = grid

      queue


      Here are my experiments, and none seem to work correctly, especially if the value has a white space.

      If the env variable has a whitespace in value
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE 0,KEY1=VALUE1\""
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out

      ————————————————————————————————————————————————————————————————————————————
      No whitespace in value
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=VALUE0,KEY1=VALUE1\""
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      "PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      KEY1=VALUE1"
      KEY0=VALUE0

      The first env variable name has “ and last env value has “ in the end

      ————————————————————————————————————————————————————————————————————————————
      Values are single quoted
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1"
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out

      Nothing was set
      ————————————————————————————————————————————————————————————————————————————
      Values are escaped double quotes
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0=\"VALUE 0\",KEY1=VALUE1"

      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      [vahi@hpc-jak condor-glite]$

      ————————————————————————————————————————————————————————————————————————————
      MYENV is within quotes, and env variable values are in single quotes
      +remote_cerequirements = "PASSENV==1 && WALLTIME==600 && PRIORITY==50 && MYENV==\"PEGASUSHOME=/auto/rcf-40/vahi/SOFTWARE/pegasus/pegasus-4.4.0cvs,KEY0='VALUE 0',KEY1=VALUE1\""

      [vahi@hpc-jak condor-glite]$ grep PEGASUSHOME test.out
      [vahi@hpc-jak condor-glite]$ grep KEY test.out
      ————————————————————————————————————————————————————————————————————————————

        Attachments

          Activity

            People

            • Assignee:
              mayani Rajiv Mayani
              Reporter:
              vahi Karan Vahi
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: