<div dir="ltr">Hi Dennis,<div><br></div><div>Thank you again for the script. That has helped me a lot. I had a follow-up question on that. For large datasets, it is easy to read-in only the required levels and time period to reduce memory usage. Hence is it possible to club the following two lines together</div><div><br></div><div><div><span style="white-space:pre"> </span>u_all<span style="white-space:pre"> </span>= addfiles_GetVarShort (filesu, "u")</div><div><span style="white-space:pre"> </span>u<span style="white-space:pre"> </span>= u_all(iStrt:iLast,{lev1:lev2},:,:)</div></div><div><br></div><div>Thank you,</div><div>Bappaditya Nag<br></div><div class="gmail_extra"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Post-doctoral Research Scholar<br><div>Earth, Ocean and Atmospheric Science<br></div><div>Florida State University<br></div>Ph. No. : +91-8697458936<br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, Aug 2, 2018 at 10:37 AM, Dennis Shea <span dir="ltr"><<a href="mailto:shea@ucar.edu" target="_blank">shea@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>[1] <b>If necessary</b>, NCL's <a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml" target="_blank"> <b>addfiles</b></a> will rebase the 'time' to the time units on the initial file. <br>[2] <b>addfiles</b> will not apply the scale_factor and add_offset. The <a href="http://www.ncl.ucar.edu/Document/Functions/Contributed/short2flt.shtml" target="_blank">short2flt</a> function will use the initial scale_factor and add_offset only.<br></div><br>I had an old function (<b>addfiles_getVarShort</b>) that does just what Toni suggested. Offline, I sent it to Bappa. Luckily, it worked :-)<br></div><div><span class=""><br><b>load</b> "./addfiles_GetVarShort.ncl"<br> <br> diri = "./"<br></span> fili =<b> <a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/systemfunc.shtml" target="_blank">systemfunc</a></b>("cd " + diri + "; ls ERA_Interim_20*.nc")<br> pthi = diri+fili<br><br> print(pthi)<br> print("=======================<wbr>==============================<wbr>========")<br><br> varName = "WhatEverVariable"<br> x = <b>addfiles_GetVarShort</b> (pthi, varName)<br> printVarSummary(x)<br> printMinMax(x,0)<br><br> print("=======================<wbr>==============================<wbr>========")<br> ymdh = <b><a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/cd_calendar.shtml" target="_blank">cd_calendar</a>(</b>x&time, -3)<br> print(x&time+" "+ymdh) ; show times<br><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jul 29, 2018 at 11:18 PM, Toni Klemm <span dir="ltr"><<a href="mailto:toni-klemm@tamu.edu" target="_blank">toni-klemm@tamu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Hi <span style="background-color:rgba(255,255,255,0)">Bappaditya,</span><div><br></div><div>Have you tried creating a new, empty array of the same size of all the files you want to concatenate and then create a loop in which you read in a single file, convert it to from short to float and then “add” it to your array?</div><div><br></div><div>I’m not familiar with your data, so my idea might not work for you. Good luck though!</div><div><br></div><div>Greetings from Texas,</div><div>Toni</div><div><br></div><div><br><div id="m_-4132942325421571567m_4092569752632533779AppleMailSignature"><b style="background-color:rgba(255,255,255,0);font-size:13pt">Toni Klemm</b><b style="background-color:rgba(255,255,255,0);font-size:13pt"><span>, </span><span>Ph.D.</span></b><div><span style="background-color:rgba(255,255,255,0);font-size:13pt">Postdoctoral Research Associate</span></div><div><span style="background-color:rgba(255,255,255,0);font-size:13pt">Department of Ecosystem Science and Management</span></div><div><span style="background-color:rgba(255,255,255,0);font-size:13pt">Texas A&M University, College Station, TX</span></div><div><span style="background-color:rgba(255,255,255,0);font-size:13pt">Contributor at the </span><a href="https://www.eccforum.org/" style="background-color:rgba(255,255,255,0);font-size:13pt" target="_blank">Early Career Climate Forum</a></div><div><span style="background-color:rgba(255,255,255,0);font-size:13pt"><a href="http://www.toni-klemm.de/" target="_blank">www.toni-klemm.de</a> | </span><a href="https://twitter.com/ToniKlemm" style="background-color:rgba(255,255,255,0);font-size:13pt" target="_blank">@toniklemm</a></div><div><br></div></div><div><div class="m_-4132942325421571567h5"><div><br>On Jul 30, 2018, at 12:00 AM, Bappaditya Nag <<a href="mailto:bappaditya.nag82@gmail.com" target="_blank">bappaditya.nag82@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px">Hi,</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px"><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px">I have 6 hrly ERA-Interim data stored in chunks of monthly files in short format (i.e., each file have their own scale and offset factor). When I read such multiple files in ncl (i.e., using v->f[:]) and convert to float (i.e., using short2flt), I do not get the correct result since the scale and offset of the first file is used for all the files.</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px"><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px">Solution : I can unpack and repack the files in a single file, but that is against the purpose of storing the data in optimal compact format. I am looking for a solution to read every file, convert to float and then concatenate the data. Can this be done? Or any other more effcient solution. Any help is appreciated.</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px"><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px">Present Code is as follows:</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px"><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial;margin-top:0px;margin-bottom:0px"><span style="font-size:12pt">files</span><span style="font-size:12pt"> </span><span style="font-size:12pt">= systemfunc("ls " + dir + "ERA_Interim/ERA_Interim_v_"+y<wbr>ear+"*.nc")</span><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><font size="3"><span style="font-size:12pt">fil</span></font><font size="3"><span style="font-size:12pt"><span> </span></span></font><font size="3"><span style="font-size:12pt">= addfiles(files,"r")</span></font><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><font size="3"><span style="font-size:12pt">ListSetType (fil, "cat")</span></font><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><font size="3"><span style="font-size:12pt">v6hrly</span></font><font size="3"><span style="font-size:12pt"><span> </span></span></font><font size="3"><span style="font-size:12pt">= short2flt(fil[:]->v(iStrt:iLas<wbr>t,{lev1:lev2},:,:))</span></font><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><font size="3"><span style="font-size:12pt"><br></span></font></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><font size="3"><span style="font-size:12pt">Also attached is a figure generated from the code.</span></font></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial">Warm Regards,</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial">Bappaditya Nag</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial">Post-doctoral Scholar</div><div style="color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:16px;text-decoration-style:initial;text-decoration-color:initial">Florida State University</div><div><div class="m_-4132942325421571567m_4092569752632533779gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
</div></blockquote></div></div><blockquote type="cite"><div><xt_Meridional Wind.png></div></blockquote><blockquote type="cite"><div><span>______________________________<wbr>_________________</span><br><span>ncl-talk mailing list</span><br><span><a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a></span><br><span>List instructions, subscriber options, unsubscribe:</span><br><span><a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">http://mailman.ucar.edu/mailma<wbr>n/listinfo/ncl-talk</a></span><br></div></blockquote></div></div><br>______________________________<wbr>_________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailma<wbr>n/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>