Getting to Know Your LDD Repo ============================= .. toctree:: :maxdepth: 3 /ldd-repo-overview Overview ++++++++ Assuming a new :doc:`LDD Github Repo has been generated for you `, this section and its subsections will describe the directories, associated files, and and the automated dictionary generation and validate performed automatically by the repo through Github Actions. The default directory structure and file locations are very important to ensuring the continuous integration built into the repository executes successfully. ---- LDD Generation and Validation ++++++++++++++++++++++++++++++++ Each LDD Github repo generated from the `ldd-template repo `_ will contain `Github Actions `_ to do the following: Generating schemas and schematrons ---------------------------------- The github action will download and run the latest `LDDTool `_ and generate the schemas and schematrons from your Ingest LDD using the expected LDDTool flags. **What does this mean?** The schemas and schematrons in your repository should always be generated using the latest LDDTool and using the configuration that an operational release would expect. It really helps ensure all LDDs are generated the same way and include the same set of files. Additionally, *you should* **NOT** *generate and push schemas or schematrons to your repository manually*. To minimize inconsistencies across LDDs, all schemas and schematrons posted on Github should be auto-generated by Github, or generated and pushed by `EN Operations staff `_. **What does this NOT mean?** This does not mean you shouldn't `download LDDTool `_ and test locally. Github Actions are great, but can be kind of slow generating the schemas, so I would recommend running LDDTool locally to test before pushing an update to Github per the :doc:`LDD Update Process `. LDD Validation --------------- The github action also downloads `Validate Tool `_, performs basic validation of your schemas and schematrons, and performs regression testing against any tests in the ``test/`` directory of your repo. See :doc:`LDD Regression Testing ` for more info on how to populate the ``test`` directory. ---- Repo Directory Structure ++++++++++++++++++++++++ .. warning:: Please do not **remove** or **move** the directories unless told it is OK by the descriptions of the directories below. Below is an example of the default repo layout for the **cart** namespace. This is an example and the exact layout of the generated repo is subject to change, but the following subsections of this guide will be updated as needed. For the latest layout checkout the `ldd-template repo `_:: ├── README.md ├── pds4_versions.txt ├── build/ │   └── development/ │   └── release/ ├── docs/ │   └── source/ │   └── Makefile │   └── examplenamespace.pdf │   └── make.bat │   └── requirements.txt ├── logs/ ├── src/ │   ├── PDS4_EXAMPLE_IngestLDD.xml │   └── dependencies/ ├── test/ │   ├── No.Data │   ├── test1_FAIL.xml │   └── test1_VALID.xml README.md --------- A template README is included in the repository with minimal information provided. Update anything contained within curly braces (``{````}``). pds4_versions.txt ----------------- This text file contains the applicable PDS4 Versions to build against for LDDs in your repo. For Discipline LDDs, this file is automatically updated after each system build by the PDSEN Operations Team with the newly released PDS4 Information Model. For Mission LDDs, this file must be manually updated whenever a new PDS4 Information Model version is needed. .. note:: Feel free to remove versions that are not applicable or add new versions by adding a newline and the applicable version number. Note: The Github CI/CD only works for PDS4 IM versions 1.11.0.0 and later. build/ ------ This directory is where all the schemas, schematrons and other files generated from LDDTool are placed. * ``development/`` - this directory will contain the current development version of the schema. You should always be able to direct someone here if they want to be able to test with the schemas that are currently in development for next build. * ``release/`` - this directory will contain all released versions of the LDD from PDS4 Stadard Version 1.15.0.0 and later. This directory is maintained for **configuration management only**. Users should be directed to download from the archival, released version at `pds.nasa.gov `_. docs/ ------ Contains the template documentation (in Markdown) and the necessary build software and configuration for Sphinx to auto-generate the online documentation for this LDD. An overview of the content within this directory: * ``source/`` - contains the template documentation files, in Markdown, used to generate the online documentation for your repo, as well as the Sphinx configuration file, `conf.py `_. The updates needed in that file should be self-documented in your repo, or see the `latest version `_ in the ldd-template repo. You will want to update the remaining Markdown documentation files accordingly per your repository and the guidelines from the Stewards' Handbook. * ``Makefile`` - this is used to auto-generate the online documentation using Sphinx. You should not need to touch this. * ``examplenamespace.pdf`` - this is the auto-generated PDF from the Markdown documentation under the `source/` directory. Once you update the ``Project`` value in your Sphinx configuration file, this filename will change to your LDDs name. `Here is where that value comes from in the template repository `_. * ``make.bat`` - similar to the ``Makefile``, this is used to auto-generate the online documentation using Sphinx. You should not need to touch this. * ``requirements.txt`` - this is used to build the software used to auto-generate the online documentation using Sphinx. You should not need to touch this. logs/ ---- Contains the logs from the last attempt to auto-generate the LDDs. src/ ---- This directory is where the IngestLDD and its associated dependencies are located. * ``PDS4__IngestLDD.xml`` - this file must be located in this directory in order for the continuous integration to work properly. The file name should follow the syntax of ``PDS4_NAMESPACE_IngestLDD.xml`` where ``NAMESPACE`` is the shortened namespace for your LDD, e.g. ``GEOM``, ``IMG``, ``MARS2020``. * ``dependencies/`` - in the event that your Discipline Dictionary inherits classes from another Discipline Dictionary, this directory will use Github submodules to link to the other repository. If that does not apply to your dictionary, you can remove this directory. If it does, contact the PDS Steward Admins to get help setting this up. test/ ----- Directory containing regression tests for the LDD. See :doc:`LDD Regression Testing ` for more details. ---- Trying It Out +++++++++++++ Before trying it out on your own repository, it is recommended that you review the `LDD Update Process `_ and completing the `LDD Update and Build Tutorial `_ ---- Handling LDD dependencies +++++++++++++++++++++++++ If your LDD has a dependency on another LDD, you will need to complete the following changes to your repository after it has been created: 1. Create a new ``src/dependencies`` directory:: mkdir -p src/dependencies 2. Create a submodule to the LDD is in the dependent repo and set the branch to pull from:: cd src/dependencies git submodule add https://github.com/pds-data-dictionaries/ldd-geom git submodule set-branch --branch main src/dependencies/ldd-geom/ 3. You should now see a ``src/dependencies/ldd_geom`` directory:: $ ls README.md ldd-geom/ 4. Commit and push your changes to a branch and merge.