WhatEvery1Says Manifest Schema Documentation

Draft v0.6
March 29, 2015

Contents

• Introduction
• Basic YAML Format
• WhatEvery1Says Manifest Field Definitions
• Index of Fields

Introduction

A WhatEvery1Says manifest is a valid YAML document saved with the suffix .yml. Manifests can be used to store information about the following:

1. Primary or processed data data files.
2. Workflow sequences.
3. Supporting materials (processing scripts, documentation, visualisations, and so on).

A typical manifest will resemble the following template (terminal nodes with [] can have a list or array as their value; otherwise, the value is assumed to be text):

Note that the sample below does not represent changes in v0.6.

  ---
      manifestId:
      namespace:
      version:
      resourceId:
      resourceType:
      title:
      altTitle:
      label:
      publicationInfo:
        publication:
        publisher:
        publicationDate:
        contentType:
        documentType:
        authors: []
        language: []
        country: []
        OCR:
        resourceMetadata:
        rights:
      collectionInfo:
        collectors: []
        collectionDate:
        accessMethod: []
        resourceLocation:
      relationships: []
      processes:
        ref:
        seq: []
      notes: []

Certain items will not appear in all manifests. For example, a manifest recording a workflow process will normally not contain collectionInfo. This documentation explains each item in detail in the manifest field definitions below.

Basic YAML Format

The example below demonstrates the basics of the YAML format.

  ---        
  manifestId:1
  namespace: WE1S
  version: 1.0
  resourceId: 1
  title: Understanding Vintage Cartoons
  publicationInfo:
    publication: The New York Times
    publicationDate: 2015
    authors:
    - seq: 1
      author: Fred Flintstone
      - seq: 2
      author: George Jetson
  ---

The above document defines an associative array with 4 top-level keys. The value of a key may itself be an array containing further key-value pairs. YAML keys are easily mapped onto the keys of data objects in most programming languages or fields in a database.

Valid YAML documents must observe the following formatting guidelines:

• YAML data structure hierarchy is maintained by outline indentation. The specific number of spaces in the indentation is unimportant as long as parallel elements have the same left justification and the hierarchically nested elements are indented further. Tab characters are never allowed for indentation.
• Key-value pairs are separated by a colon followed by a space.
• A hyhen followed by a space before a key indicates that it is an item in a list.
• A YAML stream contain multiple documents separated by ---.
• The character # is used to precede comments that are ignored by a YAML parser.

YAML auto-detects data type, so it is generally not necessary to enclose strings like "George Jetson" in quotation marks in order to distinguish them from integers or other data types. However, quotation marks can be used for clarity. See the YAML guidelines for ways to specify other data types.

Multiline values will be parsed as a single string if the has the modifier | or >. The former will preserve line spacing and white space; the latter will fold multiple lines into a single paragraph.

  # Preserve line breaks and other white space
  poem: |
    There once was a short man from Ealing
      Who got on a bus to Darjeeling
            
  # Fold line breaks into a single paragraph
  prose: >
    There once was a short man from Ealing
    Who got on a bus to Darjeeling

Items in lists do not have an inherent sequence. Therefore, if their order needs to be preserved, a key identifying the sequence number should be supplied:

  authors:
  - seq: 1
    author: Fred Flintstone
  - seq: 2
    author: George Jetson

YAML has an inline, or "flow" style similar to JSON. It can express multiple values as a list, enclosed in square brackets, or as an array of key-value pairs, enclosed in curly brackets. Both list and array items should be separated by a comma followed by a space. Hence the sample manifest above can also be represented as follows:

  [{manifestId: 1, namespace: WE1S, version: 1.0, resourceId: 1, title: Understanding Vintage Cartoons, 
    publicationInfo: {publication: The New York Times, publicationDate: 2015, authors: [{seq: 1, author: 
    Fred Flintstone}, {seq: 2, author: George Jetson}]}]

The structure is perhaps easier to see in a typical JSON pretty-printed format:

  [
    {manifestId: 1,
     namespace: WE1S,
     version: 1.0,
     resourceId: 1,
     title: Understanding Vintage Cartoons,
     publicationInfo: {
       publication: The New York Times,
       publicationDate: 2015, 
       authors: [
         {seq: 1,
          author: Fred Flintstone
         },
         {seq: 2,
          author: George Jetson
         }
       ]
     }
   ]

Flow notation can be useful for compact data transfer, but it may also be required for filling out forms. For instance, Bolt does not possess a "repeatable" content type for items in a list. Therefore, a user may have to enter multiple items as a list.

WhatEvery1Says Manifest Field Definitions

Borrowing from database terminology, each YAML key node will generally be referred to as "fields". Some fields, such as relationships and notes, are not required. Depending on the type of information stored, collectionInfo and processes may be optional.

Unlike XML, YAML does not have to declare a root note explicitly, although top-level fields can be said to be "contained by" the root field. If an explicit naming of the root is required (such as for conversion to XML), it can be done so by concatenating the namespace and version values. For example:

root: WE1S1.0
  namespace: WE1S
  version: 1.0

Detailed descriptions of the individual fields are given below.

accessMethod:
  url: http://somewebsite.com
  api: http://somewebresource.com/?search=somequery

authors:
- author:
  firstname: Fred
  lastname: Flintstone
- author:
  firstname: George
  lastname: Jetson

  authors:
  - Fred Flintsone
  - George Jetson
  
  authors:
  - seq: 1
    author: Fred Flintstone
  - seq: 2
    author: George Jetson

collectors:
- collector:
  firstname: Fred
  lastname: Flintstone
- collector:
  firstname: George
  lastname: Jetson

  collectors:
  - Fred Flintsone
  - George Jetson
  
  collectors:
  - seq: 1
    collector: Fred Flintstone
  - seq: 2
    collector: George Jetson

notes:
  -note: This is a note.
  -note: This is a second note.

processes:
  -seq: 1
   action: Removed stop words.
   date: 2015-01-03
   resp: Scott Kleinman 
   processedText: /path/to/file/without/stopwords   

resourceMetadata:
  namespace: http://dublincore.org/documents/dcmi-namespace/
  prefix: DC
  field: type

process:
  type: collection