% $Id$

%=============================================================================
% Requirements may be itemized under a main topic:
%=============================================================================
%=============================================================================

The following are requirements for I/O operations and are in addition
to the applicable general ESMF requirements (see ESMF General 
Requirements document, section 8.1.3).

\req{General I/O requirements}

\sreq{Establish ESMF metadata conventions}

A set of metadata conventions on gridded data on structured and
unstructured grids, as well as observational data has to be determined
within ESMF. The ESMF metadata conventions shall be based on the
NetCDF Climate-Forecast (CF) Metadata Conventions
\cite{NetCDF_CF}, extending it for unstructured gridded data
as well as for observational data defined on location streams.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI, NSIPP, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Document inspection \\
\end{reqlist}

\sreq{Provide automatic generation of metadata}

As an option, provide automatic generation of metadata for gridded
data on structured and unstructured grids, as well as observational
data covered by ESMF metadata conventions. This requirement shall
cover all supported formats.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test \\
\end{reqlist}


\sreq{Provide generation of companion metadata file}

Upon request, a companion metadata file, including the specified ESMF
metadata, shall be provided for all supported formats.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test \\
\end{reqlist}

\sreq{Provide generation of GrADS control file}

Upon request, a companion GrADS file, including the specified ESMF
metadata, shall be provided for IEEE binary and GRIB files.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test \\
\end{reqlist}




\req{I/O requirements for supporting different data formats}
%-----------------------------------------------------------------------------

\sreq{Reading and writing netCDF files for structured gridded data}

ESMF has to provide interface and software to read and write netCDF
files for structured gridded data covered by ESMF metadata conventions. 

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NSIPP, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}


\sreq{Reading and writing netCDF files on unstructured grids}

ESMF has to provide interface and software to read and write netCDF
files for data on unstructured grids covered by ESMF metadata conventions. 

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO, NSIPP, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}


\sreq{Reading and writing netCDF files for observational data}

ESMF has to provide interface and software to read and write netCDF
files for observational data  covered by ESMF metadata conventions. 

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO, NSIPP, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}


\sreq{Reading and writing netCDF files using DODS}

ESMF has to provide interface and software to read and write netCDF
files for data covered by ESMF metadata conventions over the Internet, 
using DODS.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NSIPP, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
{\bf Notes:} The I/O ought to be extensible to other URI based
systems. \\
\end{reqlist}

\sreq{Reading and writing HDF4 files for structured gridded data}

ESMF has to provide interfaces and software to read and write HDF4 files for 
structured gridded data covered by ESMF metadata conventions.
 

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}



\sreq{Reading and writing HDF4 files for data on unstructured grids}

ESMF has to provide interfaces and software to read and write HDF4
files for data on unstructured grids covered by ESMF metadata conventions.
 

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}

\sreq{Reading and writing HDF4 files for observational data}

ESMF has to provide interfaces and software to read and write HDF4
files for observational data  covered by ESMF metadata conventions.
 

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}

\sreq{Reading and writing HDF5 files for structured gridded data}

ESMF has to provide interfaces and software to read and write HDF5
files for structured gridded data covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. 
\end{reqlist}



\sreq{Reading and writing HDF5 files for data on unstructured grids}

ESMF has to provide interfaces and software to read and write HDF5 for data 
on unstructured grids covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. 
\end{reqlist}


\sreq{Reading and writing HDF5 files for observational data}

ESMF has to provide interfaces and software to read and write HDF5
files for data for observational data  covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. 
\end{reqlist}



\sreq{Reading and writing HDF-EOS files for structured gridded data}

ESMF has to provide interfaces and software to read and write HDF-EOS
files for structured gridded data covered by ESMF metadata conventions.


\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}


\sreq{Reading and writing HDF-EOS files for data on unstructured grids}

ESMF has to provide interfaces and software to read and write HDF-EOS
files for data on unstructured grids covered by ESMF metadata conventions.


\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}


\sreq{Reading and writing HDF-EOS files for observational data}

ESMF has to provide interfaces and software to read and write HDF-EOS
files for observational data  covered by ESMF metadata conventions.


\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
\end{reqlist}



\sreq{Reading and writing GRIB files for structured gridded data}

ESMF has to provide interfaces and software to read and write GRIB
files for structured gridded data covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} NCEP-GSM, NCEP-SSI \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
{\bf Notes:} Although GRIB1 is currently in operational use at NCEP,
it is proposed that ESMF will support GRIB2 format assuming transition
in the next two years.
\end{reqlist}


\sreq{Reading and writing GRIB files for data on unstructured grids}

ESMF has to provide interfaces and software to read and write GRIB
files for data on unstructured grids covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 3 \\
{\bf Source:} \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
{\bf Notes:} Although GRIB1 is currently in operational use at NCEP,
it is proposed that ESMF will support GRIB2 format assuming transition
in the next two years.
\end{reqlist}


\sreq{Reading and writing native binary files}

ESMF has to provide interfaces and software to read and write native
binary files for structured gridded data, data on unstructured grids, and 
observational data  covered by ESMF metadata conventions, for each machine 
accompanied by XML metadata for data covered by ESMF metadata conventions,
assuming that the computer architecture supports the IEEE 754 Floating
Point Standard. 

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI, NSIPP, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
{\bf Notes:} XML metadata should be specified. Possibility of BIG
ENDIAN and LITTLE ENDIAN byte ordering should be taken into account.
A byte swap and specified byte style facility is needed (MIT).
\end{reqlist}


\sreq{Reading and writing BUFR files}

ESMF has to provide interfaces and software to read and write BUFR files for
observational data covered by ESMF metadata conventions.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-SSI (milestone) \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test. \\
{\bf Notes:} Limitations on types of data, etc.
\end{reqlist}


\req{Parallel I/O requirements}


\sreq{Single-threaded I/O of distributed data}

ESMF should provide single-threaded I/O of distributed data when a
single process acquires all the data from other processors and writes
them to the disk or read data from the disk and passes them to other 
processes. 


\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-GSM, NCEP-SSI, NSIPP, CAM-EUL, CLM, CCSM-CPL, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test.
\end{reqlist}


\sreq{Multi-threaded I/O of distributed data to multiple files}


ESMF should provide multi-threaded I/O of distributed data when each
process reads data from and writes data to an independent file. 

This include fast reading and writing independent temporary files by 
processors using high-speed disks.

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} DAO, NCEP-SSI, NSIPP, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test.
\end{reqlist}


\sreq{Multi-threaded read of distributed data from a single file}


ESMF should provide multi-threaded read of distributed data when each
process reads data from the the same file.  

\begin{reqlist}
{\bf Priority:} 1 \\
{\bf Source:} NCEP-GSM, NCEP-SSI, DAO, NSIPP, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test.
{\bf Note} The ESMF I/O may provide this functionality only when it is 
enabled by underlying library. 
\end{reqlist}


\sreq{Multi-threaded write of distributed data to a single file}


ESMF should provide multi-threaded write of distributed data to the same 
file.  

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} NCEP-GSM, NCEP-SSI, DAO, NSIPP, CAM-EUL, CLM, CCSM-CPL, POP, CICE, MIT \\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test.
{\bf Note} The ESMF I/O may provide this functionality only when it is 
enabled by underlying library - otherwise it should provide a status
bit to indicate the operation is not possible i.e. it shouldn't crash!
\end{reqlist}

\sreq{Synchronous and asynchronous I/O}

ESMF shall provide an asynchronous option for all ESMF I/O models.
It shall provide async read and write operations, 
the capability to wait on individual or groups of I/O operations,
and query functions for the state of an operation.

\begin{reqlist}
{\bf Priority:} 2 \\
{\bf Source:} DAO, NSIPP\\
{\bf Status:} Proposed \\
{\bf Verification:} Unit test.
\end{reqlist}


