[ncl-talk] reg_multlin_stats: Y has missing values. Not allowed!

Thu Nov 16 07:08:29 MST 2017

Hi Dennis,

Thanks once more for your help, I understood what was causing the missing values issue in the first place and why that was happening in the polar latitudes lower in the troposphere.

So I changed the levels that I was applying the MLR and I had no problem.

Thanks as well for the help in terms on how to calculate the MLR reg coef as a function of latitude/height.

Much appreciated!

Regards,
Andreas

On 16 Nov 2017, at 01:29, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:

Fill an array with the desired coefficients. EGL

; create an array to hold coefficients ... then fill the array

  ncoef = 6
  bcoef = new( (/ncoef,klev,nlat/), "float")

try the attached.

Good luck

On Tue, Nov 14, 2017 at 4:09 PM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Hi Dennis,

Thanks for your help, now I got a grip of how and why the error was caused. I really appreciate your time invested into that.

The thing is that what I had in mind writing this code is that I could apply the MLR from the 1D regressors to the lev*lat variable I have, in order to plot the partial regression coefficients as a function of latitude/height.

Any idea on how to achieve that?

Cheers,
Andreas

On 14 Nov 2017, at 20:51, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:

Well, one big issues is the the most northerly latitude and all latitudes poleward of 60S were all _FillValue (missing).

 LATN   =  80.0     ; avoid regions 'loaded' with missing values
 LATS   = -60.0
...

  do nf = 0, nfili-1
     ff1    = addfile(diri+fili(nf), "r")
     wstar  = ff1->$varname$(ntStrt:ntLast,:,{LATN:LATS},0) ; 'lon' is degenerate; ignore
                                                            ; ff1->$varname$(ntStrt:ntLast,:,:,:)
[SNIP]

====
I reworked the code 'a bit.'

Your 'double do' creates a multiple linear regression for each iteration

  nf = 0
  do gg = 0,klev-1
      do hh = 0,nlat-1
         ;test = reg_multlin_stats(nies3(:,gg,hh),NIES2,0)    ; original code
          test = reg_multlin_stats(WSS(nf,:,gg,hh),NIES2,0)
          printVarSummary(test)
      end do
  end do

Good Luck
D

On Thu, Nov 9, 2017 at 6:50 AM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Hi Dennis,

I’ve played around with the code trying to find a way to tackle the missing values problem, so I interpolated my array across every dimension to fill the missing values with normal values.

Now I get the error:

fatal:divide: Division by 0, Can't continue
fatal:Div: operator failed, can't continue

The error now is caused in line 14687 in the contributed.ncl under the reg_multlin function once more. If you want to replicate the previous error you can just use as Y (dependent variable) the nies variable I’m using in my script.

I’ve put the data and the script In the ftp as per your instructions. You can find them under the names:

pp_wstar_refC1SD.zip (input diri in my code)
regressors_refC1SD.zip (input diri2 in my code)
mlr.ncl

As you can see although the input files are 6 I’m only trying to do the MLR for the first one (NIES).

Cheers,
Andreas

On 9 Nov 2017, at 02:59, Dennis Shea <shea at ucar.edu<mailto:shea at ucar.edu>> wrote:

Hello,

I can see where the error message came from:

function reg_multlin_stats(Y[*]:numeric, XP:numeric, opt)
[SNIP]
       if (isatt(Y,"_FillValue")  .and. any(ismissing(Y))) then
           print("reg_multlin_stats: Y has missing values. Not allowed!")
           exit
       end if
[SNIP]

The documentation for the underlying fortran subroutine says missing values are allowed. So, maybe I should not have that in the NCL interface. The interface subsequently calls:

https://www.ncl.ucar.edu/Document/Functions/Built-in/reg_multlin.shtml

===========================

Can you please send a clean, 'minimalistic' script that yields the error message?

Small files can be attached to the reply. If the data file is large, please use ftp:

ftp ftp.cgd.ucar.edu<http://ftp.cgd.ucar.edu/>
anonymous
your_email
cd incoming
put ....
put ...
quit

After successful transfer, send an email indicating the name(s) of the files.

Regards

On Wed, Nov 8, 2017 at 7:10 AM, Andreas Chrysanthou <eeac at leeds.ac.uk<mailto:eeac at leeds.ac.uk>> wrote:
Hi NCL users,

I’ve bumped into some missing values in my independent variable while trying to do a multiple linear regression analysis with reg_multin_stats function.

I want to produce lat/height plots (levxlat) of the predictors for the 31 years of my time series after the MLR as well as look at the Yest from those in terms of its spatial patterns.

The error I’m getting is "reg_multlin_stats: Y has missing values. Not allowed!”

PrintVarSummary of the variables I’m using is just below, whereas a snippet of the script is following after that.

Any ideas on how to get round this problem?

Variable: WSTAR
Type: float
Number of Dimensions: 4
Dimensions and sizes: [model | 6] x [year | 31] x [lev | 31] x [lat | 64]
standardize_op_ncl : dim_standardize_n over dimension(s): year
NCL : month_to_annual
actual_range : ( -0.6522911, 1e+20 )

Variable: NIES2
Type: float
Number of Dimensions: 2
Dimensions and sizes: [year | 31] x [reg_index | 5]
Number Of Attributes: 3
_FillValue : 1e+20
NCL : month_to_annual
standardize_op_ncl : dim_standardize_n over dimension(s): year

NIES1  = new ( (/5,31/), "float" )
    NIES1(0,:) = reg(0,{1980:2010},0)
    NIES1(1,:) = reg(0,{1980:2010},1)
    NIES1(2,:) = reg(0,{1980:2010},2)
    NIES1(3,:) = reg(0,{1980:2010},3)
    NIES1(4,:) = reg(0,{1980:2010},4)

NIES1!0 = "reg_index"
NIES2   = NIES1(year|:,reg_index|:)

nies = WSTAR(0,:,:,:)
test_ = test(0,:,:)

do gg = 0,dimsizes(WS_&lev)-1
    do hh = 0,dimsizes(WS_&lat)-1
        test_(gg,hh) = reg_multlin_stats(nies(:,gg,hh),NIES2,0)
    end do
end do

Cheers,
Andreas
---

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

<mlr.ncl>

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

<mlr.ncl>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171116/39910a96/attachment.html>