[Met_help] [rt.rap.ucar.edu #82581] History for MODE v6.0 - object file text format

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I just switched to version 6.0 of MET from version 5.2. I'm using MODE. I
noticed that in the output object text files (i.e., *_obj.txt) some of the
numerical entries are suddenly appearing in scientific notation, whereas I
had never seen that when running older versions of MODE. It is causing my
Fortran code to hiccup when trying to read in such data.

Is scientific notation a new entry for output formatting in version 6.0, or
is it just very rare and I just happened to have never seen it until now?
Can scientific notation be shut off? It's not just that. As shown in the
attached screenshot of an output file, the lengths of the output values can
differ, and when the value is a whole number it gets printed as a pure
integer rather than a float.

I would like to make a suggestion - in future versions of MODE, have the
output spat out using a consistent format (like %8.6f for attributes that
will always be limited to 0.0 - 1.0, for example).

Jeff Duda

-- 
Jeff Duda
Post-doctoral research fellow
University of Oklahoma School of Meteorology


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MODE v6.0 - object file text format
From: Jeff Duda
Time: Tue Oct 31 12:54:40 2017

It would probably help to mention the main column in the output that
is
causing me problems is AREA_RATIO. While there is variability in the
length
of the values in other columns, it is much more variable in this
particular
column for some reason.

Jeff Duda

On Tue, Oct 31, 2017 at 1:32 PM, met_help at ucar.edu via RT
<met_help at ucar.edu
> wrote:

> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
>         "MODE v6.0 - object file text format",
> a summary of which appears below.
>
> There is no need to reply to this message right now.  Your ticket
has been
> assigned an ID of [rt.rap.ucar.edu #82581].
>
> Please include the string:
>
>          [rt.rap.ucar.edu #82581]
>
> in the subject line of all future correspondence about this issue.
To do
> so,
> you may reply to this message.
>
> For more information, please see:
>
> MET Online Tutorial:
>
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
>
> MET Users Guide:
>    https://www.dtcenter.org/met/users/docs/overview.php
>
> MET FAQs:
>    https://www.dtcenter.org/met/users/support/faqs/index.php
>
> MET-Help Email Archive:
>    http://mailman.ucar.edu/pipermail/met_help
>
>                         Thank you,
>                         met_help at ucar.edu
>
>
-------------------------------------------------------------------------
> I just switched to version 6.0 of MET from version 5.2. I'm using
MODE. I
> noticed that in the output object text files (i.e., *_obj.txt) some
of the
> numerical entries are suddenly appearing in scientific notation,
whereas I
> had never seen that when running older versions of MODE. It is
causing my
> Fortran code to hiccup when trying to read in such data.
>
> Is scientific notation a new entry for output formatting in version
6.0, or
> is it just very rare and I just happened to have never seen it until
now?
> Can scientific notation be shut off? It's not just that. As shown in
the
> attached screenshot of an output file, the lengths of the output
values can
> differ, and when the value is a whole number it gets printed as a
pure
> integer rather than a float.
>
> I would like to make a suggestion - in future versions of MODE, have
the
> output spat out using a consistent format (like %8.6f for attributes
that
> will always be limited to 0.0 - 1.0, for example).
>
> Jeff Duda
>
> --
> Jeff Duda
> Post-doctoral research fellow
> University of Oklahoma School of Meteorology
>
>


--
Jeff Duda
Post-doctoral research fellow
University of Oklahoma School of Meteorology

------------------------------------------------
Subject: MODE v6.0 - object file text format
From: John Halley Gotway
Time: Tue Nov 14 15:53:05 2017

Hi Jeff,

Sorry for the delay in responding.  We received your email regarding
the use of scientific notation and writing a fixed-width output
format.

The reason that we don't use a Fortran-like fixed width format is that
the header columns of MET contain user-settable strings.  Columns in
the output STAT files like MODEL, OBTYPE, and DESC can be set by the
user to be of arbitrary length.  So we don't know ahead of time what a
reasonable fixed width for each column would be.

The AsciiTable class in MET does a lot of work to line up the output
columns, making sure there's at least one space between them.

Additionally, the amount of precision written is also user-settable.
The "output_precision" config file entry can be changed from its
default value of 5 decimal places... up to 12 decimal places.  That
too would impact the fixed width format.

Ultimately, the formatting of the numbers that the AsciiTable class
writes is handled by a call to printf.  I believe its the "%g"
formatting option which can result in scientific notation:
  http://www.cplusplus.com/reference/cstdio/printf/

I could see a config file option in the future that would tell MET to
always use "%f" instead of "%g"... to force it not to avoid scientific
notation.  If you think that'd be helpful, I could add a development
task for a future release.  Just let me know if you think that'd help.

As for the fixed-width format, I think you're out of luck.  I'd
suggest writing a script to post-process the MET output into the
fixed-width format that your Fortran code expects.

Thanks,
John Halley Gotway


------------------------------------------------
Subject: MODE v6.0 - object file text format
From: Jeff Duda
Time: Tue Nov 14 16:33:33 2017

John,
I ended up going with the latter option of manually decoding the text
to
determine which format to use, then using a custom format. It's not
pretty,
but it was my only way. Therefore, I would definitely prefer to have a
config file option as you described. Users might think it to be a
weird
setting to ask for, but you could include a note describing why it's
there
in the user's guide.

Jeff

On Tue, Nov 14, 2017 at 4:53 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Jeff,
>
> Sorry for the delay in responding.  We received your email regarding
the
> use of scientific notation and writing a fixed-width output format.
>
> The reason that we don't use a Fortran-like fixed width format is
that the
> header columns of MET contain user-settable strings.  Columns in the
output
> STAT files like MODEL, OBTYPE, and DESC can be set by the user to be
of
> arbitrary length.  So we don't know ahead of time what a reasonable
fixed
> width for each column would be.
>
> The AsciiTable class in MET does a lot of work to line up the output
> columns, making sure there's at least one space between them.
>
> Additionally, the amount of precision written is also user-settable.
The
> "output_precision" config file entry can be changed from its default
value
> of 5 decimal places... up to 12 decimal places.  That too would
impact the
> fixed width format.
>
> Ultimately, the formatting of the numbers that the AsciiTable class
writes
> is handled by a call to printf.  I believe its the "%g" formatting
option
> which can result in scientific notation:
>   http://www.cplusplus.com/reference/cstdio/printf/
>
> I could see a config file option in the future that would tell MET
to
> always use "%f" instead of "%g"... to force it not to avoid
scientific
> notation.  If you think that'd be helpful, I could add a development
task
> for a future release.  Just let me know if you think that'd help.
>
> As for the fixed-width format, I think you're out of luck.  I'd
suggest
> writing a script to post-process the MET output into the fixed-width
format
> that your Fortran code expects.
>
> Thanks,
> John Halley Gotway
>
>
>


--
Jeff Duda
Post-doctoral research fellow
University of Oklahoma School of Meteorology

------------------------------------------------
Subject: MODE v6.0 - object file text format
From: John Halley Gotway
Time: Tue Nov 14 16:49:11 2017

Jeff,

OK, thanks for getting back to me.  As a followup, would you prefer
the
simplicity of a true/false flag...

    use_scientific_notation = TRUE; // or FALSE

Or the power (and complexity) of explicitly specifying the printf
format
for floating point numbers...

    output_float_format = "%15.5f";

The latter would give you the flexibility of defining a field width...
granting you more control of the output column widths.

Thanks,
John





On Tue, Nov 14, 2017 at 4:33 PM, Jeff Duda via RT <met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82581 >
>
> John,
> I ended up going with the latter option of manually decoding the
text to
> determine which format to use, then using a custom format. It's not
pretty,
> but it was my only way. Therefore, I would definitely prefer to have
a
> config file option as you described. Users might think it to be a
weird
> setting to ask for, but you could include a note describing why it's
there
> in the user's guide.
>
> Jeff
>
> On Tue, Nov 14, 2017 at 4:53 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Jeff,
> >
> > Sorry for the delay in responding.  We received your email
regarding the
> > use of scientific notation and writing a fixed-width output
format.
> >
> > The reason that we don't use a Fortran-like fixed width format is
that
> the
> > header columns of MET contain user-settable strings.  Columns in
the
> output
> > STAT files like MODEL, OBTYPE, and DESC can be set by the user to
be of
> > arbitrary length.  So we don't know ahead of time what a
reasonable fixed
> > width for each column would be.
> >
> > The AsciiTable class in MET does a lot of work to line up the
output
> > columns, making sure there's at least one space between them.
> >
> > Additionally, the amount of precision written is also user-
settable.  The
> > "output_precision" config file entry can be changed from its
default
> value
> > of 5 decimal places... up to 12 decimal places.  That too would
impact
> the
> > fixed width format.
> >
> > Ultimately, the formatting of the numbers that the AsciiTable
class
> writes
> > is handled by a call to printf.  I believe its the "%g" formatting
option
> > which can result in scientific notation:
> >   http://www.cplusplus.com/reference/cstdio/printf/
> >
> > I could see a config file option in the future that would tell MET
to
> > always use "%f" instead of "%g"... to force it not to avoid
scientific
> > notation.  If you think that'd be helpful, I could add a
development task
> > for a future release.  Just let me know if you think that'd help.
> >
> > As for the fixed-width format, I think you're out of luck.  I'd
suggest
> > writing a script to post-process the MET output into the fixed-
width
> format
> > that your Fortran code expects.
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> >
>
>
> --
> Jeff Duda
> Post-doctoral research fellow
> University of Oklahoma School of Meteorology
>
>

------------------------------------------------
Subject: MODE v6.0 - object file text format
From: Jeff Duda
Time: Tue Nov 14 20:44:36 2017

John,
For me a simple logical flag would suffice, but the formatting
statement is
more powerful. I'd say go with the flag for now. If someone else
complains
in the future, you can expand it to allow the full formatting
specification.

Jeff

On Tue, Nov 14, 2017 at 5:49 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Jeff,
>
> OK, thanks for getting back to me.  As a followup, would you prefer
the
> simplicity of a true/false flag...
>
>     use_scientific_notation = TRUE; // or FALSE
>
> Or the power (and complexity) of explicitly specifying the printf
format
> for floating point numbers...
>
>     output_float_format = "%15.5f";
>
> The latter would give you the flexibility of defining a field
width...
> granting you more control of the output column widths.
>
> Thanks,
> John
>
>
>
>
>
> On Tue, Nov 14, 2017 at 4:33 PM, Jeff Duda via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82581 >
> >
> > John,
> > I ended up going with the latter option of manually decoding the
text to
> > determine which format to use, then using a custom format. It's
not
> pretty,
> > but it was my only way. Therefore, I would definitely prefer to
have a
> > config file option as you described. Users might think it to be a
weird
> > setting to ask for, but you could include a note describing why
it's
> there
> > in the user's guide.
> >
> > Jeff
> >
> > On Tue, Nov 14, 2017 at 4:53 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hi Jeff,
> > >
> > > Sorry for the delay in responding.  We received your email
regarding
> the
> > > use of scientific notation and writing a fixed-width output
format.
> > >
> > > The reason that we don't use a Fortran-like fixed width format
is that
> > the
> > > header columns of MET contain user-settable strings.  Columns in
the
> > output
> > > STAT files like MODEL, OBTYPE, and DESC can be set by the user
to be of
> > > arbitrary length.  So we don't know ahead of time what a
reasonable
> fixed
> > > width for each column would be.
> > >
> > > The AsciiTable class in MET does a lot of work to line up the
output
> > > columns, making sure there's at least one space between them.
> > >
> > > Additionally, the amount of precision written is also user-
settable.
> The
> > > "output_precision" config file entry can be changed from its
default
> > value
> > > of 5 decimal places... up to 12 decimal places.  That too would
impact
> > the
> > > fixed width format.
> > >
> > > Ultimately, the formatting of the numbers that the AsciiTable
class
> > writes
> > > is handled by a call to printf.  I believe its the "%g"
formatting
> option
> > > which can result in scientific notation:
> > >   http://www.cplusplus.com/reference/cstdio/printf/
> > >
> > > I could see a config file option in the future that would tell
MET to
> > > always use "%f" instead of "%g"... to force it not to avoid
scientific
> > > notation.  If you think that'd be helpful, I could add a
development
> task
> > > for a future release.  Just let me know if you think that'd
help.
> > >
> > > As for the fixed-width format, I think you're out of luck.  I'd
suggest
> > > writing a script to post-process the MET output into the fixed-
width
> > format
> > > that your Fortran code expects.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > >
> > >
> >
> >
> > --
> > Jeff Duda
> > Post-doctoral research fellow
> > University of Oklahoma School of Meteorology
> >
> >
>
>


--
Jeff Duda
Post-doctoral research fellow
University of Oklahoma School of Meteorology

------------------------------------------------