[ncl-talk] Issue with reading shapefiles (NCL 6.4.0 only)

[Rick Brownrigg]

Fabien,

The issue turns out to be that the fields reported as type "none" are
actually Integer64 fields. NCL utilizes gdal/ogr to support shapefiles, and
support was introduced in ogr v2.0 for integer64 types (
https://trac.osgeo.org/gdal/wiki/rfc31_ogr_64). Prior to that, is appears
that there was some sort of automated mapping to a type that could
accommodate the actual data, i.e., to either an integer or double. I
speculate that NCL6.3.0 was built against a prior version of ogr.

I am implementing a fix in NCL, which will make it into a 6.4.1 release. In
the meantime, a work-around is to use the "ogr2ogr" command-line utility
from gdal.org to convert Integer64 to integer or double:

    ogr2ogr -mapFieldType integer64=integer dstShapefile.shp
srcShapefile.shp  # note dst/src ordering

I will send you in a separate email a copy of the France_adm2 shapefile
that has been converted.

Hope that helps...
Rick

On Tue, Apr 25, 2017 at 4:23 PM, Fabien Paulot <fabien.paulot at noaa.gov>
wrote:

> Hi Rick,
>
> thanks for the very quick reply.
> The shape file is too big to be sent by email but you can download it from
> GADM
> http://www.gadm.org/country (select shapefile as file format).
> I know this is a problem for France_adm2.shp
> Thanks
> Fabien
>
>
> On Apr 25, 2017, at 6:03 PM, Rick Brownrigg <brownrig at ucar.edu> wrote:
>
> Hi Fabien,
>
> This is a bit odd. Out of a collection of about a dozen shapefiles I had
> on hand, none of them displayed the issue for an integer field. However, I
> found one instance where a double field shows up as type "none" on 6.4.0,
> and is thus unreadable, yet is fine under 6.3.0.  I will look into it right
> away. I don't suppose you could send me your shapefile as an additional
> test case?
>
> Rick
>
>
> On Tue, Apr 25, 2017 at 3:00 PM, Fabien Paulot - NOAA Affiliate <
> fabien.paulot at noaa.gov> wrote:
>
>> Hello,
>>
>> it seems integers are not properly recognized in ncl 6.4.0 and cannot be
>> read.
>> This used to work fine
>>
>> Here is the output of ncl_filedump on the same shapefile with these two
>> versions. Note how the ID_1 and ID_2 field change
>>
>> Any idea what's going on.
>>
>> Thanks a lot
>> Fabien
>>
>> ncl_filedump /archive/Fabien.Paulot/DATA/Shapefiles/France/FRA_adm2.shp
>>
>>
>> Copyright (C) 1995-2015 - All Rights Reserved
>>  University Corporation for Atmospheric Research
>>  NCAR Command Language Version 6.3.0
>>  The use of this software is governed by a License Agreement.
>>  See http://www.ncl.ucar.edu/ for more details.
>>
>> Variable: f
>> Type: file
>> filename:    FRA_adm2
>> path:    /archive/Fabien.Paulot/DATA/Shapefiles/France/FRA_adm2.shp
>>    file global attributes:
>>       layer_name : FRA_adm2
>>       geometry_type : polygon
>>       geom_segIndex : 0
>>       geom_numSegs : 1
>>       segs_xyzIndex : 0
>>       segs_numPnts : 1
>>    dimensions:
>>       geometry = 2
>>       segments = 2
>>       num_features = 96  // unlimited
>>       num_segments = 477
>>       num_points = 248568
>>    variables:
>>       integer geometry ( num_features, geometry )
>>
>>       integer segments ( num_segments, segments )
>>
>>       double x ( num_points )
>>
>>       double y ( num_points )
>>
>>       integer ID_0 ( num_features )
>>
>>       string ISO ( num_features )
>>
>>       string NAME_0 ( num_features )
>>
>>       integer ID_1 ( num_features )
>>
>>       string NAME_1 ( num_features )
>>
>>       integer ID_2 ( num_features )
>>
>>       string NAME_2 ( num_features )
>>
>>       string HASC_2 ( num_features )
>>
>>       integer CCN_2 ( num_features )
>>
>>       string CCA_2 ( num_features )
>>
>>       string TYPE_2 ( num_features )
>>
>>       string ENGTYPE_2 ( num_features )
>>
>>       string NL_NAME_2 ( num_features )
>>
>>       string VARNAME_2 ( num_features )
>>
>>
>>  Copyright (C) 1995-2017 - All Rights Reserved
>>  University Corporation for Atmospheric Research
>>  NCAR Command Language Version 6.4.0
>>  The use of this software is governed by a License Agreement.
>>  See http://www.ncl.ucar.edu/ for more details.
>>
>> Variable: f
>> Type: file
>> filename:    FRA_adm2
>> path:    /archive/Fabien.Paulot/DATA/Shapefiles/France/FRA_adm2.shp
>>    file global attributes:
>>       layer_name : FRA_adm2
>>       geometry_type : polygon
>>       geom_segIndex : 0
>>       geom_numSegs : 1
>>       segs_xyzIndex : 0
>>       segs_numPnts : 1
>>    dimensions:
>>       geometry = 2
>>       segments = 2
>>       num_features = 96  // unlimited
>>       num_segments = 477
>>       num_points = 248568
>>    variables:
>>       integer geometry ( num_features, geometry )
>>
>>       integer segments ( num_segments, segments )
>>
>>       double x ( num_points )
>>
>>       double y ( num_points )
>>
>>       none ID_0 ( num_features )
>>
>>       string ISO ( num_features )
>>
>>       string NAME_0 ( num_features )
>>
>>       none ID_1 ( num_features )
>>
>>       string NAME_1 ( num_features )
>>
>>       none ID_2 ( num_features )
>>
>>       string NAME_2 ( num_features )
>>
>>       string HASC_2 ( num_features )
>>
>>       none CCN_2 ( num_features )
>>
>>       string CCA_2 ( num_features )
>>
>>       string TYPE_2 ( num_features )
>>
>>       string ENGTYPE_2 ( num_features )
>>
>>       string NL_NAME_2 ( num_features )
>>
>>       string VARNAME_2 ( num_features )
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170426/06474017/attachment.html