[ncl-talk] activate the NCL environment from within a shell script
Gus Correa
gus at ldeo.columbia.edu
Sun Jan 3 12:35:08 MST 2021
Hi All
I don't know if this is the problem, but anyway ...
AFAIK, cron doesn't source the initialization files in your home directory
(.bashrc, .bash_profile, etc).
You can export the specific environment variables you need explicitly in
the beginning of the script that cron runs.
Alternatively, you can source your initialization files in your script or
in your crontab,
but that may carry more environment variables than you really want in the
cron job to know about.
See for example this:
https://unix.stackexchange.com/questions/67940/cron-ignores-variables-defined-in-bashrc-and-bash-profile
I hope this helps,
Gus Correa
On Sun, Jan 3, 2021 at 12:21 PM Rick Brownrigg via ncl-talk <
ncl-talk at mailman.ucar.edu> wrote:
> HI Sam,
>
> I don't fully understand exactly the issue, as I do not have a testing
> environment to replicate what you are seeing. However, I suspect the core
> of the issue is that when the cron job is run by the system, it's not
> *your* .bashrc file that is used, rather one elsewhere on the system (I
> don't know offhand where, or if any, is used).
>
> All that "conda init" does is append these lines to the .bashrc in your
> homedir (it appended to .bash_profile on my Mac):
>
> # >>> conda initialize >>>
> # !! Contents within this block are managed by 'conda init' !!
> __conda_setup="$('/Users/brownrig/miniconda3/bin/conda' 'shell.bash'
> 'hook' 2> /dev/null)"
> if [ $? -eq 0 ]; then
> eval "$__conda_setup"
> else
> if [ -f "/Users/brownrig/miniconda3/etc/profile.d/conda.sh" ]; then
> . "/Users/brownrig/miniconda3/etc/profile.d/conda.sh"
> else
> export PATH="/Users/brownrig/miniconda3/bin:$PATH"
> fi
> fi
> unset __conda_setup
> # <<< conda initialize <<<
>
> Perhaps placing this into your bash script before the call to "conda
> activate". Minimally the lines in red are enough to set up the environment.
> There may be a better way to do this. Perhaps invoking bash from cron with
> "--rcfile /path/to/your/.bashrc"
>
> Hope that helps...
> Rick
>
>
>
> On Thu, Dec 31, 2020 at 12:31 PM Sam McClatchie via ncl-talk <
> ncl-talk at mailman.ucar.edu> wrote:
>
>> Colleagues
>>
>> [I am using Ubuntu 18.04]
>>
>> I want to call an NCL script from a bash shell script. Normally I would
>> just activate the environment from the command line:
>>
>> (base) smcc at Jaguar:~$ conda activate ncl_stable
>> (ncl_stable) smcc at Jaguar:~$
>>
>> and then run the shell script in the terminal:
>>
>> ./download_latest_GHRSST_data_from_ERDdap.sh
>>
>> This works fine.
>>
>> The problem is that I want to automate by using a shell script called in
>> a cron job. I want to incorporate this NCL script in a cron job
>> containing other calls to R, so being able to activate the NCL environment
>> from within a shell script would be helpful. I suspect others have done
>> something similar, so I'd appreciate your guidance.
>>
>> ### details follow #########################
>>
>> When I try to activate the NCL environment from a bash shell script using:
>>
>> /home/smcc/anaconda3/bin/conda activate ncl_stable
>>
>> in the shell script, the following error is returned:
>>
>> (base) smcc at Jaguar:/mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/shell$
>> ./download_latest_GHRSST_data_from_ERDdap.sh
>>
>> Error message start ---------------------------
>>
>> CommandNotFoundError: Your shell has not been properly configured to use
>> 'conda activate'.
>> To initialize your shell, run
>>
>> $ conda init <SHELL_NAME>
>>
>> Currently supported shells are:
>> - bash
>> - fish
>> - tcsh
>> - xonsh
>> - zsh
>> - powershell
>>
>> See 'conda init --help' for more information and options.
>>
>> IMPORTANT: You may need to close and restart your shell after running
>> 'conda init'.
>>
>> ./download_latest_GHRSST_data_from_ERDdap.sh: line 19: ncl: command not
>> found
>> (base) smcc at Jaguar:/mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/shell$
>>
>> Error message end --------------------------------------------
>>
>> Running conda init bash before calling the shell script does not fix the
>> problem.
>>
>> (ncl_stable) smcc at Jaguar:/mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/shell$
>> conda init bash
>> no change /home/smcc/anaconda3/condabin/conda
>> no change /home/smcc/anaconda3/bin/conda
>> no change /home/smcc/anaconda3/bin/conda-env
>> no change /home/smcc/anaconda3/bin/activate
>> no change /home/smcc/anaconda3/bin/deactivate
>> no change /home/smcc/anaconda3/etc/profile.d/conda.sh
>> no change /home/smcc/anaconda3/etc/fish/conf.d/conda.fish
>> no change /home/smcc/anaconda3/shell/condabin/Conda.psm1
>> no change /home/smcc/anaconda3/shell/condabin/conda-hook.ps1
>> no change
>> /home/smcc/anaconda3/lib/python3.7/site-packages/xontrib/conda.xsh
>> no change /home/smcc/anaconda3/etc/profile.d/conda.csh
>> no change /home/smcc/.bashrc
>> No action taken.
>>
>> Another recommendation was to prepend a line to the bash script:
>>
>> eval "$(command conda 'shell.bash' 'hook' 2> /dev/null)"
>>
>> but this doesn't seem to work when I call the shell script from cron.
>>
>> My cron file is just one line calling the shell script to run
>> interactively:
>>
>> 0 12 * * * bash -i
>> /mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/shell/download_latest_GHRSST_data_from_ERDdap.sh
>>
>> and here is my shell script:
>>
>> start shell script ------------------------------
>>
>> eval "$(command conda 'shell.bash' 'hook' 2> /dev/null)"
>> #!/bin/bash
>> ## download_latest_GHRSST_data_from_ERDdap.sh
>> # NOTE: for this script to work, call the script in cron using:
>> # "bash -i ./download_latest_GHRSST_data_from_ERDdap.sh"
>> # to run the script in interactive mode so the .bashrc file is sourced
>> # Otherwise conda activate will not work
>>
>> # Use crontab -e to edit the cron file
>> # (see https://linux4one.com/how-to-set-cron-jobs-on-ubuntu-18-04/)
>> # use cronitor to troubleshoot cron file
>> # (see https://cronitor.io/cron-reference/cron-troubleshooting-guide).
>> # Try $ cronitor select to test run
>>
>> ########################################
>> # download the latest GHRSST data from erddap
>> ########################################
>> # Rscript
>> "/mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/R/download_GHRSST_MUR-JPL-L4-GLOB-v4.1_daily.R"
>>
>> ########################################
>> # plot GHRSST data
>> ########################################
>> # activate the ncl environment
>> conda activate ncl_stable
>>
>> ncl
>> /mnt/data/dynamic_data/projects/projects2020/recreational_fishing_GHRSST/ncl/plot_daily_GHRSST-MUR-JPL-L4-GLOB-v4_1_around_New_Zealand.ncl
>>
>> end shell script -------------------------------
>>
>> Best fishes
>>
>> Sam
>> --
>> Sam McClatchie (fisheries oceanographer)
>> & Elena Turin (accounting & auditing)
>> FishOcean Enterprises www.fishocean.info
>> 38 Upland Rd, Huia, Auckland 0604, New Zealand
>> cell: 027 752 8495
>>
>>
>> "The time has come", the tui said,
>> "to talk of many things:
>> Of songs - and ferns - and flowering flax,
>> of pukekos and dreams ..."
>>
>> (not Lewis Carroll)
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at mailman.ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at mailman.ucar.edu
> List instructions, subscriber options, unsubscribe:
> https://mailman.ucar.edu/mailman/listinfo/ncl-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20210103/8c7d398c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SamMcClatchie2_small.jpg
Type: image/jpeg
Size: 45362 bytes
Desc: not available
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20210103/8c7d398c/attachment.jpg>
More information about the ncl-talk
mailing list