Getting to Know Your LDD Repo

Overview

Assuming a new LDD Github Repo has been generated for you, this section and its subsections will describe the directories, associated files, and and the automated dictionary generation and validate performed automatically by the repo through Github Actions.

The default directory structure and file locations are very important to ensuring the continuous integration built into the repository executes successfully.

LDD Generation and Validation

Each LDD Github repo generated from the ldd-template repo will contain Github Actions to do the following:

Generating schemas and schematrons

The github action will download and run the latest LDDTool and generate the schemas and schematrons from your Ingest LDD using the expected LDDTool flags.

What does this mean?

The schemas and schematrons in your repository should always be generated using the latest LDDTool and using the configuration that an operational release would expect. It really helps ensure all LDDs are generated the same way and include the same set of files.

Additionally, you should NOT generate and push schemas or schematrons to your repository manually. To minimize inconsistencies across LDDs, all schemas and schematrons posted on Github should be auto-generated by Github, or generated and pushed by EN Operations staff.

What does this NOT mean?

This does not mean you shouldn’t download LDDTool and test locally. Github Actions are great, but can be kind of slow generating the schemas, so I would recommend running LDDTool locally to test before pushing an update to Github per the LDD Update Process.

LDD Validation

The github action also downloads Validate Tool, performs basic validation of your schemas and schematrons, and performs regression testing against any tests in the test/ directory of your repo. See LDD Regression Testing for more info on how to populate the test directory.

Repo Directory Structure

Warning

Please do not remove or move the directories unless told it is OK by the descriptions of the directories below.

Below is an example of the default repo layout for the cart namespace. This is an example and the exact layout of the generated repo is subject to change, but the following subsections of this guide will be updated as needed. For the latest layout checkout the ldd-template repo:

├── README.md
├── pds4_versions.txt
├── build/
│   └── development/
│   └── release/
├── docs/
│   └── source/
│   └── Makefile
│   └── examplenamespace.pdf
│   └── make.bat
│   └── requirements.txt
├── logs/
├── src/
│   ├── PDS4_EXAMPLE_IngestLDD.xml
│   └── dependencies/
├── test/
│   ├── No.Data
│   ├── test1_FAIL.xml
│   └── test1_VALID.xml

README.md

A template README is included in the repository with minimal information provided. Update anything contained within curly braces ({``}``).

pds4_versions.txt

This text file contains the applicable PDS4 Versions to build against for LDDs in your repo.

For Discipline LDDs, this file is automatically updated after each system build by the PDSEN Operations Team with the newly released PDS4 Information Model.

For Mission LDDs, this file must be manually updated whenever a new PDS4 Information Model version is needed.

Note

Feel free to remove versions that are not applicable or add new versions by adding a newline and the applicable version number. Note: The Github CI/CD only works for PDS4 IM versions 1.11.0.0 and later.

build/

This directory is where all the schemas, schematrons and other files generated from LDDTool are placed.

  • development/ - this directory will contain the current development version of the schema. You should always be able to direct someone here if they want to be able to test with the schemas that are currently in development for next build.

  • release/ - this directory will contain all released versions of the LDD from PDS4 Stadard Version 1.15.0.0 and later. This directory is maintained for configuration management only. Users should be directed to download from the archival, released version at pds.nasa.gov.

docs/

Contains the template documentation (in Markdown) and the necessary build software and configuration for Sphinx to auto-generate the online documentation for this LDD. An overview of the content within this directory:

  • source/ - contains the template documentation files, in Markdown, used to generate the online documentation for your repo, as well as the Sphinx configuration file, conf.py. The updates needed in that file should be self-documented in your repo, or see the latest version in the ldd-template repo. You will want to update the remaining Markdown documentation files accordingly per your repository and the guidelines from the Stewards’ Handbook.

  • Makefile - this is used to auto-generate the online documentation using Sphinx. You should not need to touch this.

  • examplenamespace.pdf - this is the auto-generated PDF from the Markdown documentation under the source/ directory. Once you update the Project value in your Sphinx configuration file, this filename will change to your LDDs name. Here is where that value comes from in the template repository.

  • make.bat - similar to the Makefile, this is used to auto-generate the online documentation using Sphinx. You should not need to touch this.

  • requirements.txt - this is used to build the software used to auto-generate the online documentation using Sphinx. You should not need to touch this.

logs/

Contains the logs from the last attempt to auto-generate the LDDs.

src/

This directory is where the IngestLDD and its associated dependencies are located.

  • PDS4_<NAMESPACE>_IngestLDD.xml - this file must be located in this directory in order for the continuous integration to work properly. The file name should follow the syntax of PDS4_NAMESPACE_IngestLDD.xml where NAMESPACE is the shortened namespace for your LDD, e.g. GEOM, IMG, MARS2020.

  • dependencies/ - in the event that your Discipline Dictionary inherits classes from another Discipline Dictionary, this directory will use Github submodules to link to the other repository. If that does not apply to your dictionary, you can remove this directory. If it does, contact the PDS Steward Admins to get help setting this up.

test/

Directory containing regression tests for the LDD. See LDD Regression Testing for more details.

Trying It Out

Before trying it out on your own repository, it is recommended that you review the LDD Update Process and completing the LDD Update and Build Tutorial

Handling LDD dependencies

If your LDD has a dependency on another LDD, you will need to complete the following changes to your repository after it has been created:

  1. Create a new src/dependencies directory:

    mkdir -p src/dependencies

  2. Create a submodule to the LDD is in the dependent repo and set the branch to pull from:

    cd src/dependencies
git submodule add https://github.com/pds-data-dictionaries/ldd-geom
git submodule set-branch --branch main src/dependencies/ldd-geom/

  3. You should now see a src/dependencies/ldd_geom directory:

    $ ls
README.md
ldd-geom/

  4. Commit and push your changes to a branch and merge.