[ncl-talk] Skipping Missing Files [SEC=UNCLASSIFIED]

[Griffith Young]

Hello Barry,
                "I assume that Guido's approach will be better for this." - I agree

                "Of course, ... sometimes my download fails ... .  I am looking for a way to download grib files from NCEP without missing part of the file (sometimes)."

                Sometimes I find it is better to use "curl" or "wget" as they have settings to cope with retries and errors:
                                $ man curl
                                $ man wget

                Then you can do something like:
                                for f in a b c ; do
                                wget http://www.example.org/directory/sub/${f}
                                if [ $? -ne 0 ] ; then echo "wget failed with non-zero return code" ; continue ; fi
                                if [ ! -f ${f} ] ; then echo "file ${f} does not exist" ; continue ; fi
                                ncl f=\"${f}\" << EOF
                                print("f: " + f)
                                EOF
                                done

                Knowledge is Power.  Your call of course.

Regards,
                Griff.


From: Barry Lynn [mailto:barry.h.lynn at gmail.com]
Sent: Monday, 19 December 2016 2:13 PM
To: Griffith Young
Cc: ncl-talk
Subject: Re: [ncl-talk] Skipping Missing Files [SEC=UNCLASSIFIED]

Griff:

Thank you for your suggestion.

I will need that particular NCL script to continue to plot data beyond the time of the missing file.  I assume that Guido's approach will be better for this.

Of course, the elephant in the room issue here is that sometimes my download fails even though the file is not missing or incomplete.  I am looking for a way to download grib files from NCEP without missing part of the file (sometimes).

Barry

On Mon, Dec 19, 2016 at 1:56 AM, Griffith Young <griffith.young at bom.gov.au<mailto:griffith.young at bom.gov.au>> wrote:
Hello Barry,
                An alternative, but possibly less cool method, is to call an NCL script from bash (or your preferred shell) for each file.

                The effect is then such that NCL can succeed or fail, but your script continues.
                                for f in a b c ; do
                                ncl f=\"$f\" << EOF
                                print("f: " + f)
                                EOF
                                done

                I prefer Guido's suggestion, but when it comes to quick and dirty, this is also an option.

Regards,
                Griff.

From: ncl-talk-bounces at ucar.edu<mailto:ncl-talk-bounces at ucar.edu> [mailto:ncl-talk-bounces at ucar.edu<mailto:ncl-talk-bounces at ucar.edu>] On Behalf Of Guido Cioni
Sent: Sunday, 18 December 2016 4:05 AM
To: Barry Lynn
Cc: ncl-talk
Subject: Re: [ncl-talk] Skipping Missing Files

Hey Barry,
An easy way of doing it would be to encapsulate all the file operations into an IF statement which uses either the isfilevar (http://www.ncl.ucar.edu/Document/Functions/Built-in/isfilevar.shtml) function or the isfilepresent (http://www.ncl.ucar.edu/Document/Functions/Built-in/isfilepresent.shtml) function to check the existence of either variable in a file or the file itself. You just have to put an OR so that the operations are skipped, but the script is not interrupted, if either the variable or the file are not found.
Cheers

Il 17 dic 2016 5:29 PM, "Barry Lynn" <barry.h.lynn at gmail.com<mailto:barry.h.lynn at gmail.com>> ha scritto:
Hi:

I am downloading a lot of data and processing it, file by file.

It happens that sometimes a file is missing, or a even a variable within the file is missing.

How can I tell NCL to skip the read, instead of a returning an error and ending the program.

Basically, I want NCL to note errors, and bypass them, instead of terminating.

Barry

--
Barry H. Lynn, Ph.D
Senior Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581

C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525<tel:02%20930%209525>
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
US 914 432 3108<tel:(914)%20432-3108>

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk



--
Barry H. Lynn, Ph.D
Senior Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581

C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
US 914 432 3108
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20161219/30aa9ba2/attachment.html