Qore Mapper Module Reference  1.0
 All Classes Namespaces Functions Variables Groups Pages
Mapper Module

Mapper Module Introduction

This module provides classes that help with structured data mapping, meaning the transformation of data in one or more input formats to a different output format.

Classes provided by this module:

Mapper Examples

The following is an example map hash with comments:

const DataMap = (
# output field: "id" mapper from the "Id" element of any "^attributes^" hash in the input record
"id": "^attributes^.Id",
# output field: "name": maps from an input field with the same name (no translations are made)
"name": True,
# output field: "explicit_count": maps from the input "Count" field, if any value is present then it is converted to an integer
"explicit_count": ("type": "int", "name": "Count"),
# output field: "implicit_count": runs the given code on the input record and retuns the result, the code returns the number of "Products" sub-records
"implicit_count": int sub (any $ignored, hash $rec) { return $rec.Products.size(); },
# output field: "order_date": converts the "OrderDate" string input field to a date in the specified format
"order_date": ("name": "OrderDate", "date_format": "DD.MM.YYYY HH:mm:SS.us"),
);

If this map is applied to the following data in the following way:

const MapInput = ((
"^attributes^": ("Id": 1),
"name": "John Smith",
"Count": 1,
"OrderDate": "02.01.2014 10:37:45.103948",
"Products": ((
"ProductName": "Widget 1",
"Quantity": 1,
),
)), (
"^attributes^": ("Id": 2),
"name": "Steve Austin",
"Count": 2,
"OrderDate": "04.01.2014 19:21:08.882634",
"Products": ((
"ProductName": "Widget X",
"Quantity": 4,
), (
"ProductName": "Widget 2",
"Quantity": 2,
),
)),
);
my Mapper $map(DataMap);
my list $l = $map.mapAll(MapInput);
printf("%N\n", $l);

The result will be:

list: (2 elements)
  [0]=hash: (5 members)
    id : 1
    name : "John Smith"
    explicit_count : 1
    implicit_count : 1
    order_date : 2014-01-02 10:37:45.103948 Thu +01:00 (CET)
  [1]=hash: (5 members)
    id : 2
    name : "Steve Austin"
    explicit_count : 2
    implicit_count : 2
    order_date : 2014-01-04 19:21:08.882634 Sat +01:00 (CET))

Mapper Specification Format

The mapper hash is made up of target (ie output) field names as the key values assigned to field specifications as follows:

  • True: this is a shortcut meaning map from an input field with the same name
  • a closure or call reference: meaning map from a field of the same name an apply the given code to give the value for the mapping; the closure or call reference must accept the following arguments:
    • any $value: the input field value (with the same name as the output field; to use a different name, see the code hash option below)
    • hash $rec: the current input record
  • a hash describing the mapping; the following keys are all optional (an empty hash means map from an input field with the same name with no translations):
    • "name": the value of this key gives the name of the input field; only use this if the input record name is different than the output field name; note that if this value contains "." characters and the allow_dot option is not set (see Mapper Options), then the value will be treated like "struct"
    • "struct": the value of this key gives the location of the input field in an input hash in dot notation, ex: "element.name" would ook for the field's value in the "name" key of the "element" hash in the input record
    • "code": a closure or call reference to process the field data
    • "date_format": gives the format for converting a string to a date; see date_formatting for the format of this string; note that this also implies "type" = "date"
    • "maxlen": an integer giving the maximum output string field length in bytes
    • "trunc": assign to boolean True if the field should be truncated if over the maximum field length; this key can only be set to True if the "maxlen" key is also given
    • "mand": assign to boolean True if the field is mandatory and an exception should be thrown if no input data is supplied
    • "type": this gives the output field type, can be:
      • "number": field accepts only numeric values (any non-numeric values on input will cause an exception to be thrown when mapping)
      • "integer": fields accepts only integer values (any non-integer values on input will cause an exception to be thrown when mapping; note: also "int" is accepted as an alias for "integer")
      • "date": date field

Mapper Options

Mapper objects accept the following options in the option hash:

  • "enc": the output character encoding; if not present then "UTF-8" is assumed
  • "name": the name of the mapper for use in logging and error strings
  • "trunc": an optional closure or call reference for a callback for field truncation warnings; must accept a single string giving the warning text
  • "trunc_all": if True (as evaluated by parse_boolean()) then any field without a "trunc" key (see Mapper Specification Format "trunc" description) will automatically be truncated if a "maxlen" attribute is set for the field
  • "allow_dot": if True (as evaluated by parse_boolean()) then field names with "." characters do not imply a structured internal element lookup; in this case input field names may have "." characters in them, use the "struct" key to use structured internal element loopups (see Mapper Specification Format "struct" docs for more info)
  • "input_log": an optional input data logging callback; must accept a hash giving the input data hash
  • "output_log": an optional output data logging callback; must accept a hash giving the output data hash

Release Notes

Mapper v1.0

Initial release.