-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major
-
Affects Version/s: master, 5.0.1
-
Component/s: CLI: pegasus-transfer
-
None
I tried configuring my workflows to use GO as the transfer protocol. I had no issues creating a directory on Summit, but when the local directory creation job tried to run using the CARC endpoint, I got the following error:
Traceback (most recent call last):
File "/home/shock-ssd/scottcal/pegasus/default/lib64/python3.6/site-packages/Pegasus/cli/pegasus-globus-online.py", line 202, in mkdir
transfer_client.operation_mkdir(request["endpoint"], path)
File "/home/shock-ssd/scottcal/pegasus/default/lib64/pegasus/externals/python/globus_sdk/transfer/client.py", line 964, in operation_mkdir
return self.post(resource_path, json_body=json_body, params=params)
File "/home/shock-ssd/scottcal/pegasus/default/lib64/pegasus/externals/python/globus_sdk/base.py", line 214, in post
retry_401=retry_401)
File "/home/shock-ssd/scottcal/pegasus/default/lib64/pegasus/externals/python/globus_sdk/base.py", line 387, in _request
raise self.error_class(r)
globus_sdk.exc.TransferAPIError: (403, 'ConsentRequired', 'Missing required data_access consent', 'uJMrBxANz')
I had previously authenticated using the Globus website to both the OLCF DTN and the USC CARC User Directories endpoints, and then ran pegasus-globus-online-init, went to the website, and entered the code. (BTW, I ran into an error in pegasus-globus-online-init.py; config.set() expects strings but some of the values were ints, so I just wrapped them all in str()). However, it seems like the authentication isn't working for some reason. Since the example on the documentation uses the hash notation, I used the extremely human-readable "u_aieoitmpvnbchfh6dhsxvaiarq#b92e5d86-0d7b-11eb-8938-0a5521ff3f4b" for the CARC endpoint (it's the 'Legacy Name' on the endpoint info page). This particular job is /home/shock/scottcal/runs/USC_Integrated_dax/run_7218/dags/scottcal/pegasus/CyberShake_Integrated_USC/20211113T234036-0800/create_dir_CyberShake_Integrated_USC_0_local.sub.
Thanks for your help!