Hi Karan,

I wanted to ask you about a complication we've been dealing with since
some recent python, and condor upgrades.

Our use case is that the pegasus executables (pegasus-plan etc.) are
installed by the sysadmin in /usr using (I think) `dnf install
pegasus`. We then run code installed in some virtualenv or conda env,
which includes the pegasus-wms.api package. That environment will be
running a "modern" version of python (3.9+), whereas RHEL 8 linux is
using an old python (3.6) which is what pegasus tools at that level
see.

The divergence between these python versions means that if we run
pegasus-plan without setting the PEGASUS_PYTHON variable, we will
get a PYTHONPATH that includes a mix of python3.9+ packages and
python3.6 packages in /usr, and these are now too divergent to be at
all compatible.

If, however, we set PEGASUS_PYTHON=/usr/bin/python3 then the pegasus
dependencies installed in $PEGASUS_HOME/lib64/pegasus/externals are
not in the PYTHONPATH, and the code fails.

The only workaround I've found for this is to create a standalone
environment, conda/pip install pegasus dependencies, install pegasus
into it from source and then point PEGASUS_PYTHON to that. Or try to
manually identify and get sys-admins to install pegasus dependencies
from lib64/pegasus/externals into the standard lib path.

What we want is to be able to use pegasus tools installed in /usr,
while isolating the python version pegasus needs from everything else.
What's the best way to do that? My hunch is that we need some new flag
that would say:

• I want the systemwide version of python3 for pegasus (/usr/bin/python3)
• I do not want to see the systemwide python paths for anything
  except pegasus
• I do need PEGASUS_HOME/lib64/pegasus/externals ... or I need pegasus
  to install its dependencies in a way that we can use them.

Does this make sense?

Thanks!
Ian