Skip to content

Survey ERA systems and their data-shape quirks for UDM mapping #50

Description

@nate-layman

Context

The UDM is meant to be the destination shape. Whether it actually pays off depends on how readable the source shape is at each institution. InfoEd is a working example of the problem: every meaningful field is an integer foreign key to a lookup table, so reading the raw export tells you nothing without joining ten reference tables first. The data is there; it's just locked behind indirection.

If we're going to ship adapter patterns, vignettes, or AI-assisted mapping (#related work in the lakehouse), we need a survey of what shapes we'll actually encounter in the wild.

The InfoEd-shaped problem

  • Every status, type, role, organization, and category is a numeric ID.
  • Resolving a single proposal record to readable text requires joining 5-15 lookup tables.
  • Lookup table names and IDs aren't stable across InfoEd versions or institutional configurations.
  • This isn't unique to InfoEd; Kuali Research, PeopleSoft Grants, and Workday Grants all do versions of the same thing.

The cost: even a skilled SQL writer can't quickly answer questions against the source. Mapping to the UDM has to happen first, which is exactly why the UDM exists.

ERA systems worth surveying

This is the list to start with. We need at least one practitioner from each to document the source-table shape, the lookup quirks, and the gotchas.

Major commercial ERAs

  • InfoEd / Cayuse Research Suite (post-acquisition; the suite includes proposal development, IRB, IACUC, etc.)
  • Cayuse stand-alone (Cayuse SP, Cayuse 424, Cayuse Outgoing Subaward)
  • Huron Click Portal
  • Streamlyne Research

Open-source / academic-led

  • Kuali Research (the Foundation's evolution of MIT's Coeus)

General-purpose ERPs with grants modules

  • Oracle PeopleSoft Grants Module
  • Workday Grants Management
  • Banner Research / Banner Finance with research extensions
  • SAP Grants Management (less common in higher-ed but present)

Sponsor-side and adjacent

  • Grants.gov and research.gov (federal submission portals; institutions integrate)
  • eRA Commons (NIH)
  • NSPIRES (NASA)
  • ERA Commons reporting layer for federal pass-through reporting

Homegrown / custom

  • Salesforce-based RA tools (used at several large institutions)
  • SharePoint / Access mongrels (more common than we'd like to admit)

What we want for each

For each system on the list, a short writeup capturing:

  1. Source data shape — relational? document store? are there exportable tables?
  2. Lookup-table tax — how much indirection sits between the raw row and a human-readable record?
  3. Stable vs. configurable identifiers — which fields are universal, which vary by institution?
  4. Awkward joins — patterns the UDM mapper should expect (polymorphic FKs, soft deletes, end-dated history rows, etc.)
  5. Documentation availability — is there an ERD or data dictionary anyone outside the vendor can read?
  6. Gotchas — anything that would make a UDM adapter painful.

How to use the survey

  • Use the writeups to prioritize which adapter patterns to publish first (highest enrollment + worst data shape = most leverage).
  • Feed common gaps and naming collisions into the synonym fields on udm_schema.json so the LLM-assisted mapping (smart adapters) starts ahead of the curve.
  • Identify which ERAs have stable enough data shapes that a published adapter could ship out of the box.

How to contribute

Drop a comment with the ERA you've worked with and as much of the six-item writeup above as you can answer. Even partial answers help.

Related issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions