Git Product home page Git Product logo

cedstandards / ceds-data-warehouse Goto Github PK

View Code? Open in Web Editor NEW
19.0 21.0 3.0 12.64 MB

Modeled for longitudinal storage and reporting of P-20W data, the Common Education Data Standards (CEDS) Data Warehouse implements star schema data warehouse normalization techniques for improved query performance.

Home Page: https://github.com/CEDStandards/CEDS-Data-Warehouse

License: Apache License 2.0

TSQL 100.00%
ceds education-data-standards education-data education-database sql-server data-warehouse data-warehousing data-warehouses

ceds-data-warehouse's People

Contributors

aemandreahall avatar aemduanebrown avatar aemnathanclinton avatar lkommuri avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ceds-data-warehouse's Issues

README indicates CEDS-Elements database script is within the Data Warehouse repo

Describe the bug
The README for this repo tells users to create the CEDS Elements database by running a script that is no longer in the Data Warehouse repo. Instead, it appears users must clone the CEDS-Elements repo and create that database before finished the CEDS DW set up.

To Reproduce

The database is empty by default. Proceed with the following instructions to load CEDS Element metadata and CEDS Option Set values into the Dimension tables.

Open the Create CEDS Elements V11.0.0.0 document at /src/dimension-data/CEDS-Elements-V11.0.0.0
Connect to the server on which you created the CEDS Version 11.0.0.0 instance
Run the script. This will create the CEDS-Elements-V11.0.0.0 Database

Expected behavior
The README should be accurate to the steps.

Double check that all Junk Dimension tables are being populated

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Double check that all Junk Dimension tables are being populated.

Use Case Overview
It appears the following are Junk Dimensions but are not part of the Junk Dimension population script:

DimTitleIStatuses
DimReponsibleOrganization Types
DimPsFamilyStatuses
DimPsCourseStatuses
DimPsCitizenshipStatuses
DimK12OrganizationStatuses
DimIncidentBehaviors

Correct these and check for any others.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

Junk-Table-Dimension-Population-V9.1.0.0 issue

SELECT DISTINCT t.Name, t.XMLName, t.GlobalID, c.Code, c.Description, c.CodeDefinition, NULL, NULL
FROM CEDS.dbo.Term t
JOIN CEDS.dbo.TermxCodeSet tcs
	ON t.TermID = tcs.TermID
JOIN CEDS.dbo.CodeSet cs
	ON tcs.CodeSetID = cs.CodeSetID
JOIN CEDS.dbo.CodeSetxCode csxc
	ON cs.CodeSetID = csxc.CodeSetID
JOIN CEDS.dbo.Code c
	ON csxc.CodeID = c.CodeID
WHERE t.Version = @CEDSVersion
	AND t.HasCodeSet = '1'

refers to a non-existent CEDS database. These should probably all be references to the CEDS domain in the current database:

SELECT DISTINCT t.Name, t.XMLName, t.GlobalID, c.Code, c.Description, c.CodeDefinition, NULL, NULL
FROM CEDS.Term t
JOIN CEDS.TermxCodeSet tcs
	ON t.TermID = tcs.TermID
JOIN CEDS.CodeSet cs
	ON tcs.CodeSetID = cs.CodeSetID
JOIN CEDS.CodeSetxCode csxc
	ON cs.CodeSetID = csxc.CodeSetID
JOIN CEDS.Code c
	ON csxc.CodeID = c.CodeID
WHERE t.Version = @CEDSVersion
	AND t.HasCodeSet = '1'

Update FactPSStudentEnrollments grain information to include the DimPSInstitutionStatus as a BK

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Kim Carlson

Authoring Organization(s)
South Dakota Department of Education

Email Address
[email protected]

Use Case Title
Provide a concise description that defines the use case.

Update FactPSStudentEnrollments grain information to include the DimPSInstitutionStatus as a BK

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.

South Dakota Department of Education is using National Student Clearinghouse data to populate FactPsStudentEnrollments table. We are not a P20W SLDS which means we do not have access to Post-Secondary data other than what we receive from NSCH through the StudentTracker data file.

South Dakota does not have data to correctly map Academic Term Designator so it was mapped to MISSING.

We are experiencing post-secondary schools that are changing their institution status during the school year. In this case, they went from a 2 year school to a 4 year school. When we did a little digging, it does seem like the school changed from a 2 year to 4 year school.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

This is causing discard because FactPsStudentEnrollments is seeing this as a duplicate. SD doesn't want to form some business logic to pick an enrollment since both cases are correct.

Components Affected
Delete everything from the list below except the components affected:
CEDS Data Warehouse (complete DW Design Overview Table)

No DDL in the model should be affected. Only the Bus Architecture documentation that designates the Grain/BK for the table.

Enhancements from the Data Warehouse Expansion Project V9.1.0.0

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS Team & Data Warehouse Expansion Project Workgroup
Email Address
[email protected]
Use Case Title
Provide a concise description that defines the use case.
Enhancements from the Data Warehouse Expansion Project V9.1.0.0
Use Case Overview
This update includes the following changes to the Warehouse (Note: Additional documentation is provided in the release):

  1. Add new dimension and fact tables designed by the data warehouse expansion project working group around the three uses cases (adult education, special education, NCES census poverty measure)
  2. Provide metadata linking dimension table column names to their corresponding CEDS element technical name and Global ID in the extended properties of the column. Some Fact table columns are included as well where applicable.
  3. Rename columns to better align with their corresponding CEDS technical name.
  4. Provide additional scripts to populate junk dimensions, primarily around the 3 new use cases.
  5. Provide documentation for each Fact Table indicating the grain and definition for the table.

Pull Request Number(s) (If applicable)

Use Case Background
The new fact and dimension tables are being created to support the work done through the CEDS Data Warehouse Expansion Project. The column name changes/updates are done to create better consistency and alignment with the actual CEDS element technical names.

Remove deprecated IDS Id columns from staging tables

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Johnny Cunningham
Authoring Organization(s)
CEDS
Email Address
[email protected]
Remove deprecated IDS Id columns from staging tables

Use Case Overview
The columns are no longer used by the migration process and add confusion for anyone using those staging tables.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)
Remove the following columns from the defined staging tables

Assessment.AssessmentId
Assessment.AssessmentAdministrationId
Assessment.AssessmentSubtestId
Assessment.AssessmentFormId
Assessment.AssessmentPerformanceLevelId
Assessment.DataCollectionId
AssessmentResult.DataCollectionId
CharterSchoolAuthorizer.CharterSchoolId
CharterSchoolAuthorizer.CharterSchoolAuthorizingOrganizationOrganizationId
CharterSchoolManagementOrganization.CharterSchoolManagementOrganizationOrganizationId
CharterSchoolManagementOrganization.CharterSchoolId
Discipline.PersonId
Discipline.OrganizationID_LEA
Discipline.OrganizationPersonRoleId_LEA
Discipline.OrganizationID_School
Discipline.OrganizationPersonRoleId_School
Discipline.IncidentId_LEA
Discipline.IncidentId_School
K12Enrollment.PersonId
K12Enrollment.OrganizationID_LEA
K12Enrollment.OrganizationPersonRoleId_LEA
K12Enrollment.OrganizationID_School
K12Enrollment.OrganizationPersonRoleId_School
K12Enrollment.OrganizationPersonRoleRelationshipId
K12Enrollment.DataCollectionId
K12PersonRace.DataCollectionId
K12PersonRace.PersonId
K12PersonRace.PersonDemographicRaceId
K12PersonRace.OrganizationID_LEA
K12PersonRace.OrganizationID_School
K12PersonRace.RefRaceId
K12ProgramParticipation.DataCollectionId
K12ProgramParticipation.OrganizationId_LEA
K12ProgramParticipation.OrganizationId_School
K12ProgramParticipation.PersonId
K12ProgramParticipation.ProgramOrganizationId_LEA
K12ProgramParticipation.ProgramOrganizationId_School
K12ProgramParticipation.OrganizationPersonRoleId_LEA
K12ProgramParticipation.OrganizationPersonRoleId_School
K12SchoolComprehensiveSupportIdentificationType.OrganizationId
K12SchoolComprehensiveSupportIdentificationType.K12SchoolId
K12StudentAddress.DataCollectionId
K12StudentCourseSection.DataCollectionId
K12StudentCourseSection.PersonId
K12StudentCourseSection.OrganizationID_LEA
K12StudentCourseSection.OrganizationPersonRoleId_LEA
K12StudentCourseSection.OrganizationID_School
K12StudentCourseSection.OrganizationPersonRoleId_School
K12StudentCourseSection.OrganizationID_Course
K12StudentCourseSection.OrganizationID_CourseSection
K12StudentCourseSection.OrganizationPersonRoleId_CourseSection
Migrant.PersonID
Migrant.OrganizationID_LEA
Migrant.OrganizationID_School
Migrant.LEAOrganizationPersonRoleID_MigrantProgram
Migrant.LEAOrganizationID_MigrantProgram
Migrant.SchoolOrganizationPersonRoleID_MigrantProgram
Migrant.SchoolOrganizationID_MigrantProgram
Migrant.PersonProgramParticipationId
Migrant.ProgramParticipationMigrantId
OrganizationAddress.RefStateId
OrganizationAddress.OrganizationId
OrganizationAddress.LocationId
OrganizationCalendarSession.DataCollectionId
OrganizationCalendarSession.OrganizationId
OrganizationCalendarSession.OrganizationCalendarId
OrganizationCalendarSession.OrganizationCalendarSessionId
OrganizationCustomSchoolIndicatorStatusType.DataCollectionId
OrganizationFederalFunding.DataCollectionId
OrganizationGradeOffered.DataCollectionId
OrganizationGradeOffered.OrganizationId
OrganizationGradeOffered.K12SchoolGradeOfferedId
OrganizationPhone.OrganizationId
OrganizationPhone.LEA_OrganizationTelephoneId
OrganizationPhone.School_OrganizationTelephoneId
OrganizationProgramType.DataCollectionId
OrganizationProgramType.OrganizationId
OrganizationProgramType.ProgramOrganizationId
OrganizationProgramType.ProgramTypeId
OrganizationProgramType.OrganizationProgramTypeId
PersonStatus.DataCollectionId
ProgramParticipationCTE.DataCollectionID
ProgramParticipationCTE.PersonID
ProgramParticipationCTE.OrganizationID_School
ProgramParticipationCTE.OrganizationPersonRoleID_School
ProgramParticipationCTE.OrganizationPersonRoleID_CTEProgram
ProgramParticipationCTE.OrganizationID_CTEProgram
ProgramParticipationCTE.PersonProgramParticipationId
ProgramParticipationNorD.DataCollectionID
ProgramParticipationNorD.PersonID
ProgramParticipationNorD.OrganizationID_School
ProgramParticipationNorD.OrganizationID_LEA
ProgramParticipationNorD.LEAOrganizationID_Program
ProgramParticipationNorD.SchoolOrganizationID_Program
ProgramParticipationNorD.LEAOrganizationPersonRoleId_Program
ProgramParticipationNorD.SchoolOrganizationPersonRoleId_Program
ProgramParticipationNorD.PersonProgramParticipationID
ProgramParticipationSpecialEducation.DataCollectionID
ProgramParticipationSpecialEducation.PersonID
ProgramParticipationSpecialEducation.OrganizationID_School
ProgramParticipationSpecialEducation.OrganizationID_LEA
ProgramParticipationSpecialEducation.LEAOrganizationID_Program
ProgramParticipationSpecialEducation.SchoolOrganizationID_Program
ProgramParticipationSpecialEducation.LEAOrganizationPersonRoleId_Program
ProgramParticipationSpecialEducation.SchoolOrganizationPersonRoleId_Program
ProgramParticipationSpecialEducation.PersonProgramParticipationID_LEA
ProgramParticipationSpecialEducation.PersonProgramParticipationID_School
ProgramParticipationTitleI.DataCollectionID
ProgramParticipationTitleI.PersonID
ProgramParticipationTitleI.OrganizationID_LEA
ProgramParticipationTitleI.OrganizationID_School
ProgramParticipationTitleI.LEAOrganizationPersonRoleID_TitleIProgram
ProgramParticipationTitleI.LEAOrganizationID_TitleIProgram
ProgramParticipationTitleI.LEAPersonProgramParticipationId
ProgramParticipationTitleI.SchoolOrganizationID_TitleIProgram
ProgramParticipationTitleI.SchoolOrganizationPersonRoleID_TitleIProgram
ProgramParticipationTitleI.SchoolPersonProgramParticipationId
ProgramParticipationTitleI.RefTitleIIndicatorId
ProgramParticipationTitleIII.DataCollectionID
ProgramParticipationTitleIII.PersonID
ProgramParticipationTitleIII.OrganizationID_School
ProgramParticipationTitleIII.OrganizationPersonRoleID_TitleIIIProgram
ProgramParticipationTitleIII.OrganizationID_TitleIIIProgram
ProgramParticipationTitleIII.PersonProgramParticipationId
ProgramParticipationTitleIII.ImmigrationPersonStatusId
PsInstitution.OrganizationId
PsInstitution.OrganizationOperationalStatusId
PsInstitution.OperationalStatusId
PsInstitution.MostPrevalentLevelOfInstitutionId
PsInstitution.PredominantCalendarSystemId
PsPersonRace.PersonId
PsPersonRace.PersonDemographicRaceId
PsPersonRace.OrganizationId
PsPersonRace.RefRaceId
PsPersonRace.RefAcademicTermDesignatorId
PsStudentAcademicAward.OrganizationId
PsStudentAcademicAward.PersonId
PsStudentAcademicAward.OrganizationPersonRoleId
PsStudentAcademicAward.PsStudentAcademicAwardId
PsStudentAcademicRecord.DataCollectionID
PsStudentAcademicRecord.PersonId
PsStudentAcademicRecord.OrganizationId
PsStudentAcademicRecord.OrganizationPersonRoleId
PsStudentAcademicRecord.PsStudentAcademicRecordId
PsStudentEnrollment.DataCollectionId
PsStudentEnrollment.PersonId
PsStudentEnrollment.OrganizationPersonRoleId
PsStudentEnrollment.OrganizationId_PsInstitution
StateDetail.DataCollectionId
StateDetail.PersonId
StateDetail.OrganizationId

Create Fact tables to house K12 Calendar Data (Session & Date Level)

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS / Mississippi
Email Address
[email protected]
Use Case Title
Create Fact tables to house K12 Calendar Data (Session & Date Level)

Use Case Overview
Create two Fact tables, one that will contain K12 Calendar data at the session level and another that will contain K12 Calendar Data at the day (date) level.

Pull Request Number(s) (If applicable)

Use Case Background
Needed for tracking calendar information in an SEA. Used for determining days of instruction for funding.

Create new Metadata table that stores CEDS DW Version information

As SEAs continue to use the CEDS DW and build new applications on top of the warehouse, they will need a way for those applications and downstream tools to know what version of the CEDS DW they are utilizing. The came from a request of several states involved with Generate as this table would allow Generate to read the version and display it on the screen.

Add additional fields to DimAttendances and correct naming of existing columns

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Add additional fields to DimAttendances and correct naming of existing columns

Use Case Overview
The existing DimAttendances table contains 3 fields related to "Absenteeism". These are remnants from the Generate Data Warehouse for EDFacts reporting. These will be replaced with the correct CEDS element names. In addition, the remaining attendance status elements in CEDS need to be added to accurately report on daily attendance through FactK12StudentDailyAttendances.

image

Pull Request Number(s) (If applicable)

Add PersonIdentifierSSN to DimPeople

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Deborah Donovan
Authoring Organization(s)
Mississippi Department of Education
Email Address
[email protected]
Use Case Title
Add PersonIdentifierSSN to DimPeople

Use Case Overview
We need to capture SSN as an identifier type for people

Pull Request Number(s) (If applicable)
n/a
Use Case Background
Need to extend for this additional identifier type

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Warehouse Parquet (complete DW Parquet Design Overview Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)


Follow architecture similar to K12 for DimAeDemographics and the FactAeStudentEnrollments

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address

Use Case Title
Follow architecture similar to K12 for DimAeDemographics and the FactAeStudentEnrollments

Use Case Overview
For the FactK12StudentEnrollments, items like Economic Disadvantage and Military Connected are separated in multiple Dimension tables. Recommend doing the same with FactAEStudentEnrollments and removing items from DimAeDemographics in favor of more dimension tables and the reuse of existing dimension tables that contain this data.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

Add default values to all junk dimension table columns to allow for the easy, consistent addition of new columns

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
CEDS

Email Address
[email protected]

Use Case Title
Add default values to all junk dimension table columns to allow for the easy, consistent addition of new columns

Use Case Overview
Adding new columns to tables can be difficult if the new columns a NOT NULL. Adding a default value constraint to these new columns solves this problem. To remain consistent, we should set all Junk dimension column defaults to 'MISSING'.

Pull Request Number(s) (If applicable)

Request to add columns to support the grain for DimCredentials

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Richard Heck

Authoring Organization(s)
Otis Educational Systems

Email Address
[email protected]

Use Case Title
Request to add additional columns to support the grain for the DimCredentials table

Use Case Overview
For the DimCredentials table, we will need to have the Credential Identifier and Credential Identifier System elements added to support the grain.

Use Case Background
Changes are required to support the business key/grain for populating data in the table.

Components Affected

CEDS Data Warehouse (complete DW Design Overview Table)

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Add Credential Identifier and Credential Identifier System columns to the DimCredentials table

"Data Lake"-friendly schema for CEDS Data Warehouse

Author(s)
Mike Reynolds

Authoring Organization(s)
Cedar Labs, LLC

Email Address
[email protected]

Use Case Title
"Data Lake"-friendly schema for CEDS Data Warehouse

Use Case Overview
We are working with a client that wants to take advantage of a modern cloud infrastructure, and we'd like to take the CEDS DW into the cloud.

Pull Request Number(s) (If applicable)

Use Case Background
The current requirement to use MS SQLServer is overly prescriptive - especially for cloud environments. It would be nice to have a more open alternative for large implementations that can take advantage of modern scalable architectures.

Components Affected
CEDS Data Warehouse

CEDS Data Warehouse (complete DW Design Overview Table)
TBD


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.

Documentation request: Request for Grain information for Dim/Fact tables

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Richard Heck

Authoring Organization(s)
Otis Educational Systems

Email Address
[email protected]

Use Case Title
Request for Grain information for Dim/Fact tables

Use Case Overview
When loading data to the CEDS DW, it will be useful to have the grain information for each of the table to understand how to populate the objects.

Pull Request Number(s) (If applicable)

Use Case Background
N/A

Components Affected
CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Element Gap Analysis
N/A


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Requesting a list of the columns (Business Key) for each table that make up the grain.

Extend ExitOrWithdrawalType to nvarchar(max)

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Drew Bennett-Stein
Authoring Organization(s)
State of Vermont
Email Address
[email protected]
Use Case Title
Extend ExitOrWithdrawalType to nvarchar(max)

Use Case Overview
Loading ExitOrWithdrawalType

Pull Request Number(s) (If applicable)

Use Case Background
Unable to load ExitOrWithdrawalType with Description from RefExitOrWithdrawalType.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)
K12StudentEnrollment RefExitOrWithdrawalType

CEDS Data Warehouse (complete DW Design Overview Table)
K12Enrollment ExitOrWithdrawalType
CEDS Data Migrations (complete Data Migration Impact Analysis Table)
The proposed changes in the ExitOrWithdrawalType do not directly impact the current data migration strategies or processes. However, ongoing monitoring will be required to ensure compatibility with future data migrations

CEDS Element Gap Analysis
A thorough review indicates no immediate gaps in the CEDS elements concerning the ExitOrWithdrawalType. Future assessments should be conducted to maintain alignment with evolving educational data standards and requirements.

INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Enhancements to the K12StudentEnrollment.RefExitOrWithdrawalType to support extended character limits and incorporate descriptive metadata.

As-Is Diagram
Current design limits character input and lacks detailed descriptors.

Proposed To-Be Diagram
Revised design with nvarchar(max) implementation, allowing for comprehensive descriptions and improved data quality.

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
staging schema K12Enrollment
As-Is Diagram
Present schema constrains data entry, limiting the depth of information captured for ExitOrWithdrawalType.
Proposed To-Be Diagram
Updated schema in the staging area, expanding data capacity, and aligning with Integrated Data Store enhancements.

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Adjustments to the data migration protocols to accommodate the expanded data type and ensure seamless integration with existing datasets.
New Business Rule
Implement validation checks for extended ExitOrWithdrawalType data during migration.
Existing Business Rule
Maintain current data integrity checks, with added focus on the extended data fields.
Changed Business Rule
Update transformation rules to align with the expanded data type for ExitOrWithdrawalType.

Graduation working group

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Ben Silberglitt and Tori Ryan

Authoring Organization(s)
Cedar Labs and Maine DOE

Email Address
[email protected] [email protected]

Use Case Title
Graduation Data Warehouse working group

Use Case Overview
Maine would like to leverage the CEDS DW to support public-facing reporting around graduation outcomes. Currently, Maine's reporting on this domain includes some data that the CEDS DW data schema is not equipped to handle. For example, graduation outcomes are broken out into 4yr, 5yr, and 6yr cohorts. Also, outcomes are tracked by graduation, continuing, dropout, and unknown. Maine and Cedar Labs consulted with AEM, and it sounds like a new fact table will be the best approach. We'd like to form a working group with any and all interested parties, to determine what schema this fact table should have, to serve as many needs as possible.

Pull Request Number(s) (If applicable)
N/A

Use Case Background
See use case overview above

Components Affected
CEDS Data Warehouse (complete DW Design Overview Table)
CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

Correct column names in two tables

Describe the bug
Column names are misspelled in the following tables:

In FactK12StudentAssessmentResultAggregates, all columns that begin with "StandarDeviation..." need to corrected to "StandardDeviation..."

In BridgeK12StudentAssessmentAccommodations, the PK of the table should be "BridgeK12StudentAssessmentAccommodationId" instead of "FactK12StudentAssessmentAccommodationId"

Integration of Attendance Status and Truancy Data into CEDS Data Warehouse

Author(s): Drew Bennett-Stein

Authoring Organization(s): State of Vermont, Agency of Education

Email Address: [email protected]

Use Case Title: Integration of Attendance Status and Truancy Data into CEDS Data Warehouse

Use Case Overview:
This use case proposes the integration of detailed attendance status (excused/unexcused) and truancy data into the CEDS Data Warehouse. This includes the addition of new tables such as DimAttendanceStatuses and modifications to existing tables like FactK12StudentAttendanceRates or FactK12StudentDailyAttendances for capturing the number of unexcused absences.

Pull Request Number(s) (If applicable): [To be filled after initiating the pull request]

Use Case Background:
The current CEDS Data Warehouse model lacks specific data points for Attendance Status (excused/unexcused) and does not include a field for 'NumberofDaysUnexcused' in the RoleAttendance section. This data is vital for analyzing attendance patterns and addressing issues like truancy effectively.

Components Affected:

CEDS Data Warehouse (complete DW Design Overview Table)
INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes:
The proposal is to add a new table, DimAttendanceStatuses, for detailed attendance statuses, and to enhance existing tables to capture 'NumberofDaysUnexcused'.

As-Is Diagram: [Attachment Required]

Proposed To-Be Diagram: [Attachment Required]

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes:
Addition of DimAttendanceStatuses table and modifications to FactK12StudentAttendanceRates or FactK12StudentDailyAttendances to include unexcused absences data.

As-Is Diagram: [Attachment Required]

Proposed To-Be Diagram: [Attachment Required]

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes:
Introduction of new data points for attendance status and truancy.

New Business Rule:

Capture and store detailed attendance status (excused/unexcused).
Record 'NumberofDaysUnexcused' for each student.
Existing Business Rule:

[Describe the current business rules regarding attendance data management]
Changed Business Rule:

Enhancement of the existing business rule to include detailed attendance statuses and truancy data.

Add OPEID to DimPsInstitutions and make IPEDS Identifier Nullable

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
CEDS
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Add OPEID to DimPsInstitutions and make IPEDS Identifier Nullable

Use Case Overview
Add a new column to DimPSInstitutions - see ticket CEDStandards/CEDS-Elements#638

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

DimPsInstitutionStatuses - VirtualIndicatorCode/Description data mismatch

Describe the bug
The seed data for VirtualIndicatorCode/Description in [DimPsInstitutionStatuses] is different than the Element definition on the website.

To Reproduce
SELECT distinct
[VirtualIndicatorCode]
,[VirtualIndicatorDescription]
FROM [CEDS-Data-Warehouse-V10-0-0-0].[RDS].[DimPsInstitutionStatuses]

--Compare these values to the extended properties element
https://ceds.ed.gov/CEDSElementDetails.aspx?TermId=18167

Expected behavior
The [CEDS-Data-Warehouse-V10-0-0-0].[RDS].[DimPsInstitutionStatuses] seed data should have Yes/No/Missing(?) values like the element on the site instead of the current values.

Screenshots
If applicable, add screenshots to help explain your problem.

Updates required by Generate for Federal Reporting

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
Center for the Integration of IDEA Data (CIID)

Email Address
[email protected]

Use Case Title
Provide a concise description that defines the use case.
Update the CEDS Data Warehouse to include changes required for Generate

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.

Implement federal EDFacts reporting changes required by Generate and to fully align to CEDS data elements names.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.
Data warehouse changes determined after implementing the OSC version of the CEDS Data Warehouse V9 submitted by CEPI in Michigan.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.
See docs\Version 8.0.0.0 to Version 9.0.0.0 Comparison.xlsx

Add the DimTitleIIIStatuses table to the FactK12ProgramParticipations

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Add the DimTitleIIIStatuses table to the FactK12ProgramParticipations

Use Case Overview
This is for tracking Title III statuses of a student who is enrolled in any program. The specific use case need is for tracking the Title III Accountability Progress Status of a student who is enrolled in a Title III language instruction educational program for EDFacts file specification 211.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

SEAs have the need to store student addresses along with enrollment data

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
Mississippi Department of Education
Michigan Center for Education and Performance Information

Email Address
[email protected]

Use Case Title
Provide a concise description that defines the use case.
SEAs need student addresses tied to enrollment data

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.
Add a bridge table connected to FactK12StudentEnrollments that captures 1-to-many addresses for a student.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.
Student addresses are needed for location analysis by both Michigan and Mississippi.

Components Affected

CEDS Data Warehouse (complete DW Design Overview Table)
Add a new table named RDS.BridgeK12StudentEnrollmentPersonAddresses

Add Dates to DimK12Courses

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address

Use Case Title
Add Dates to DimK12Courses

Use Case Overview
DimK12Courses is a slowly changing Dimension. Add RecordStartDateTime and RecordEndDateTime to this table.
Additionally, add CourseBeginDate and CourseEndDate to the table to represent information about the course being offered. These will not be used as record management, but will instead be metadata about whether or not the course can be offered/whether it can exist in a school catalogue.

Release Data Warehouse scripts in lower version of SQL

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown

Authoring Organization(s)
CEDS

Email Address
[email protected]

Use Case Title
Release Data Warehouse scripts in lower version of SQL. Change the compatibility level to less than 150 (2019). Perhaps in the 2012 or 2016 range.

Use Case Overview
Current release script requires SQL Server Management Studio 2019. Change the compatibility level to less than 150 (2019). Perhaps in the 2012 or 2016 range.

Add CountDateId as a foreign key reference to DimDates to several Fact tables

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Lakshmana Kommuri
Authoring Organization(s)
CEPI - State Of Michigan
Email Address
[email protected]
Use Case Title
Add CountDateId as a foreign key reference to DimDates to several Fact tables.

Use Case Overview
CountDateId is needed to populate bridge tables for three Fact tables.

  • FactK12StudentEnrollments
  • FactPsStudentAcademicRecords
  • FactSpecialEducation

Pull Request Number(s) (If applicable)

Use Case Background
A specific point-in-time date is needed to capture the appropriate details from the staging for the associated Fact records in order to fill the bridge tables.

Components Affected
CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes

  • Add CountDateId column to the three fact tables
  • Make those CountDateId Foreign key references to DimDates

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Update the Staging-to-RDS data migration scripts to populate necessary bridge tables related to FactK12StudentEnrollments, FactPsStudentAcademicRecords, and FactSpecialEducation by using the new CountDateId fields to select the correct point-in-time data from the Staging tables.

Add staff-related fact and dimension tables as designated by Staff Workgroup

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
CEPI

Email Address
[email protected]

Use Case Title
Provide a concise description that defines the use case.
Add fact and dimension tables to support robust staff reporting and analytics.

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Components Affected
Delete everything from the list below except the components affected:

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

Need staging tables and staging-to-RDS ETL scripts to populate FactK12FinancialAccountBalances and FactK12FinancialAccountGeneralLedgers

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton
Duane Brown

Authoring Organization(s)
Mississippi Department of Education

Email Address
[email protected]
[email protected]

Use Case Title
Provide a concise description that defines the use case.

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.
Mississippi is going to populate these two fact tables, but currently does not have the infrastructure to do so.

Add individual Score Metric types to Staging.AssessmentResult

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Johnny Cunningham
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Add Assessment Score Metric types to Staging.AssessmentResult

Use Case Overview
This is a complementary ticket to Ticket #43 as well as the changes to RDS.FactK12StudentAssessments in v11.

Pull Request Number(s) (If applicable)

Use Case Background
Adding the individual Score Metric types will simplify the loading of the Staging.AssessmentResult table by providing the individual Score Metric types as well as the Standard Error for each Assessment rather than having to load multiple rows per student per assessment.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Add the individual Score Metric types and the standard error for each to Staging.AssessmentResult

  • AssessmentResultScoreValueRawScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorRawScore (nvarchar(35), null)

  • AssessmentResultScoreValueScaleScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorScaleScore (nvarchar(35), null)

  • AssessmentResultScoreValuePercentile (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorPercentile (nvarchar(35), null)

  • AssessmentResultScoreValueTScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorTScore (nvarchar(35), null)

  • AssessmentResultScoreValueZScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorZScore (nvarchar(35), null)

  • AssessmentResultScoreValueACTScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorACTScore (nvarchar(35), null)

  • AssessmentResultScoreValueACTSuperScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorACTSuperScore (nvarchar(35), null)

  • AssessmentResultScoreValueSATScore (nvarchar(35), null)

  • AssessmentResultScoreValueStandardErrorSATScore (nvarchar(35), null)


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Update the Staging-to-RDS data migration scripts for Assessments to use the individual Score Metric values.

Add Primary Assignment Indicator to the Data Warehouse

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Add Primary Assignment Indicator to the Data Warehouse

Use Case Overview
Add the CEDS element Primary Assignment Indicator to the Staging.StaffAssignment table. Additionally, add the element to the Dimension table it is applicable to.

K12StudentID in FactK12StudentCounts should be a FK Reference

Describe the bug
In the table FactK12StudentCounts, the column name "K12StudentId" should be a Foreign Key (FK) reference to the DimK12Students table.

To Reproduce
Steps to reproduce the behavior:
Go to RDS.FactK12StudentCounts and expand the columns and find the K12StudentId column.

Expected behavior
Change the K12StudentId to an FK referring back to the DimK12Students table.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context

Add score metrics to FactK12StudentAssessments

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Kim Carlson

Authoring Organization(s)
South Dakota Department of Education

Email Address
[email protected]

Use Case Title
Add score metrics to FactK12StudentAssessments

Use Case Overview
There are two requests for this issue...

(1) South Dakota is requesting a new metric column is added to FactK12StudentAssessments. It would be called AssessmentResultScoreValueACTSuperScore and be a (nvarchar(35), null).

(2) South Dakota is also requesting these metrics be added into FactK12StudentAssessments with the CEDS release if possible. These metrics were provided to South Dakota as possible additional metrics that could be added. One thing, for AssessmentResultScoreStandardErrorRawScore can the word value be added to be consistent with the other metric naming conventions?

• AssessmentResultScoreValueStandardErrorRawScore (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorScaleScore (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorPercentile (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorTScore (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorZScore (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorACTScore (nvarchar(35), null)
• AssessmentResultScoreValueStandardErrorSATScore (nvarchar(35), null)

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.
These changes are needed so we can include standard error in the assessment table. We also need a new metric for ACT super score. South Dakota had a meeting with Nathan and Duane requesting this an additional metric since super score can be different than ACT scores.

Components Affected
Delete everything from the list below except the components affected:

FactK12StudentAssessments is what is affected. I'm not sure if it impacts both CEDS IDS and CEDS Data Warehouse?

CEDS Integrated Data Store (complete IDS Design Overview Table)

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Element Gap Analysis
Provide link to CEDS Align Map or Connection.


INTEGRATED DATA STORE DESIGN OVERVIEW

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes
Provide a concise description of the proposed changes.

As-Is Diagram
Add as an attachment in this issue.

Proposed To-Be Diagram
Add as an attachment in this issue.


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Provide a concise description of the proposed changes.

New Business Rule
Provide a concise and comprehensive description of the new business rule.

Existing Business Rule
Provide a concise and comprehensive description of the existing business rule.

Changed Business Rule
Provide a concise and comprehensive description of the changes to be made to the business rule.

Restructure DimPersonAddresses to link into facts via Bridge tables

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Lakshmana Kommuri
Authoring Organization(s)
CEPI - State Of Michigan
Email Address
[email protected]
Use Case Title
Restructure DimPersonAddresses to link into facts via Bridge tables

Use Case Overview
DimAddresses is currently bound to the context of K12 Students because of the inclusion of AddressTypeForLearnerOrFamily. Also, there is currently no way to track multiple addresses for a student. This proposal recommends:

  • Remove AddressTypeForLearnerOrFamily from DimPersonAddresses
  • Create a Bridge table between any fact table and DimPersonAddresses
  • Add AddressTypeForLearnerOrFamily or AddressTypeForStaff to the bridge table as needed for the context of the fact table
  • Move PersonalInformationVerification to the bridge tables as needed

Pull Request Number(s) (If applicable)

Use Case Background
CEPI needs to track multiple addresses for Students as well as addresses for staff. The data warehouse model currently doesn't support this.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

  1. For DimPersonAddresses

    • Remove AddressTypeForLearnerOrFamilyCode
    • Remove PersonalInformationVerificationCode
    • Remove PersonalInformationVerificationDescription
    • Remove DoNotPublishIndicator
  2. Create a new bridge table named BridgeK12StudentEnrollmentPersonAddresses. It will include

    • FactK12StudentEnrollmentId
    • PersonAddressId
    • AddressTypeForLearnerOrFamilyCode
    • AddressTypeForLearnerOrFamilyDescription
    • PersonalInformationVerificationCode
    • PersonalInformationVerificationDescription
    • DoNotPublishIndicator
  3. Create a new bridge table named BridgeK12StudentEconomicDisadvantagePersonAddresses. It will include

    • BridgeK12StudentEconomicDisadvantagePersonAddresses
    • FactK12StudentEconomicDisadvantageId
    • PersonAddressId
    • AddressTypeForLearnerOrFamilyCode
    • AddressTypeForLearnerOrFamilyDescription
    • PersonalInformationVerificationCode
    • PersonalInformationVerificationDescription
    • DoNotPublishIndicator

As-Is Diagram
image

Proposed To-Be Diagram
image
image


DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
The script to move data from the Staging tables to FactK12StudentEnrollments need additional SQL code to populate BridgeK12StudentEnrollmentPersonAddresses.

Add support for individual program participation to FactK12StudentAssessments

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Johnny Cunningham
Authoring Organization(s)
CEDS
Email Address
[email protected]
Use Case Title
Provide a concise description that defines the use case.
Add individual program statuses to FactK12StudentAssessments
Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.
Version 11 of the data warehouse broke DimK12Demographics into individual dimensions. This task will add programs Homelessness, Economically Disadvantaged, English Learner, Migrant, and Military Connected from the old DimK12Demographics dimension as well as support for Foster Care, Immigrant, and Title I statuses in FactK12StudentAssessments.
Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.
This change is required to track a students program participation through Assessments

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)*

CEDS Data Migrations (complete Data Migration Impact Analysis Table)*

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes

  • Add HomelessnessStatusId column to the Fact table - FactK12StudentAssessments
  • Make HomelessnessStatusId a Foreign key reference to DimHomelessnessStatuses
  • Add EconomicallyDisadvantagedStatusId column to the Fact table - FactK12StudentAssessments
  • Make EconomicallyDisadvantagedStatusId a Foreign key reference to DimEconomicallyDisadvantagedStatuses
  • Add EnglishLearnerStatusId column to the Fact table - FactK12StudentAssessments
  • Make EnglishLearnerStatusId a Foreign key reference to DimEnglishLearnerStatuses
  • Add MigrantStatusId column to the Fact table - FactK12StudentAssessments
  • Make MigrantStatusId a Foreign key reference to DimMigrantStatuses
  • Add MilitaryStatusId column to the Fact table - FactK12StudentAssessments
  • Make MilitaryStatusId a Foreign key reference to DimMilitaryStatuses
  • Add FosterCareStatusId column to the Fact table - FactK12StudentAssessments
  • Make FosterCareStatusId a Foreign key reference to DimFosterCareStatuses
  • Add ImmigrantStatusId column to the Fact table - FactK12StudentAssessments
  • Make ImmigrantStatusId a Foreign key reference to DimImmigrantStatuses
  • Add TitleIStatusId column to the Fact table - FactK12StudentAssessments
  • Make TitleIStatusId a Foreign key reference to DimTitleIStatuses

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Update the Staging-to-RDS data migration scripts for Assessments to populate the new program status IDs by joining to the appropriate dimension views.

Expand the Data Warehouse in the Area of Finance

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
CEDS Data Warehouse Expansion Process Workgroup
Email Address
[email protected]
Use Case Title
Expand the Data Warehouse in the Area of Finance

Use Case Overview
The CEDS Data Warehouse Expansion Process Workgroup met from November 2021 through August 2022 and is proposing the addition of three new Fact tables and several new dimension tables to the Data Warehouse The proposed expansion timing will be aligned with the approval of several new elements and updates to existing elements within CEDS which can be found in this Elements use case ticket: CEDStandards/CEDS-Elements#493

Components Affected
CEDS Data Warehouse (complete DW Design Overview Table)

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

DW Expansion Project Finance Draft Model.pdf

Add SchoolYearId and PsDemographicId as Foreign Key to FactPsStudentAcademicAwards

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Lakshmana Kommuri
Authoring Organization(s)
CEPI - State Of Michigan
Email Address
[email protected]
Use Case Title
Add SchoolYearId and PsDemographicId as Foreign Key to FactPsStudentAcademicAwards

Use Case Overview
SchoolYearId and PsDemographicId is needed to populate the Fact table - FactPsStudentAcademicAwards

Use Case Background
CEPI needs SchoolYearId and PsDemographicId data to be available in FactPsStudentAcademicAwards.

Components Affected
CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes

  • Add SchoolYearId column to the Fact table - FactPsStudentAcademicAwards
  • Make SchoolYearId as a Foreign key references to DimSchoolYears
  • Add PsDemographicIdcolumn to the Fact table - FactPsStudentAcademicAwards
  • Make PsDemographicId as a Foreign key references to DimPsDemographics

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Update the Staging-to-RDS data migration scripts to populate the new SchoolYearId and PsDemographicId fields by joining to DimSchoolYears and vwDimPsDemographics respectively.

Version 9.1 - DimProgramTypes - Population Error

Describe the bug
The population of the DimProgramTypes table is wrong. It is populating over 20,000 options and should only be 41 options from the Program Type table.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
Populate only the 41 options from the Program Type element in this table.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Add dimensional model for Accessible Education Materials reporting use case

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
AEM Center

Email Address
[email protected]

Use Case Title
Provide a concise description that defines the use case.
Add dimensional model for Accessible Education Materials reporting

Use Case Overview
Provide a concise, high-level description of the actions of the use case. While being concise, the description should cover all possible actions for the use case.
The AEM Center is working with CEDS to identify elements and a reporting data structure to support reporting and analysis needs around accessible education materials that currently does not exist in the domain.

Pull Request Number(s) (If applicable)

Use Case Background
Provide information related to why these changes/additions are needed.

Add New Fact Table and Dimension Tables related to Salary Schedule

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Duane Brown
Authoring Organization(s)
CEDS
Email Address

Use Case Title
Add New Fact Table and Dimension Tables related to Salary Schedule

Use Case Overview
This proposal is to add a new fact to the Data Warehouse to store Salary Schedules. This is related to Elements ticket 428 (CEDStandards/CEDS-Elements#428)

image

Pull Request Number(s) (If applicable)

Use Case Background
Mississippi needs to capture/store/and report salary schedule data from their LEAs.

Remove unused functions from Staging schema

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Nathan Clinton

Authoring Organization(s)
Center for the Integration of IDEA Data -- CIID

Email Address
[email protected]

Use Case Title
Remove unused functions from Staging schema

Use Case Overview
There are several functions currently in the Staging schema that refer to CEDS IDS reference tables (Ref tables). These will not function without a fully integrated IDS installed in the same database, plus they are no longer necessary. They should be removed.

Pull Request Number(s) (If applicable)
#37

Use Case Background
These functions were created to support legacy Staging-to-IDS data migrations, which are no longer used or supported.

DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)
Description of Proposed Changes

Remove the following SQL functions

  • GetOrganizationIdentifierSystemId
  • GetOrganizationIdentifierTypeId
  • GetOrganizationRelationshipId
  • GetOrganizationTypeId
  • GetPersonIdentifierSystemId
  • GetPersonIdentifierTypeId
  • GetProgramTypeId
  • GetRefIDEAEducationalEnvironmentECId
  • GetRefIDEAEducationalEnvironmentSchoolAgeId
  • GetRefInstitutionTelephoneType
  • GetRefOrganizationLocationTypeId
  • GetRefPersonalInformationVerificationId
  • GetRefPersonIdentificationSystemId
  • GetRefStateAnsiCode
  • GetRefStateId
  • GetRoleId
  • GetStateName

Add TitleIStatusId to FactK12StudentEnrollments

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Lakshmana Kommuri
Authoring Organization(s)
CEPI - State Of Michigan
Email Address
[email protected]
Use Case Title
Add TitleIStatusId to FactK12StudentEnrollments

Use Case Overview
CEPI needed Title I Status data available in FactK12StudentEnrollments.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Migrations (complete Data Migration Impact Analysis Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)

Description of Proposed Changes

  • Add TitleIStatusId column to the Fact table - FactK12StudentEnrollments
  • Make TitleIStatusId as a Foreign key references to DimTitleIStatuses

DATA MIGRATION IMPACT ANALYSIS

Description of Proposed Changes
Update the Staging-to-RDS data migration scripts to populate the new TitleIStatusId field by joining to vwDimTitleIStatuses.

Gun Free Schools Act reporting statuses CEDS code in DimK12OrtanizationStatuses are EDFacts codes

Describe the bug
Two options within RDS.DimK12OrganizationStatuses, "YesReportingOffenses" and "YesNoReportedOffenses" are set to the EDFacts values of "YESWITHREP" and "YESWOREP" respectively. These should be updated to the CEDS codes.

Here is how these data are fixed for Generate instances:

update RDS.DimK12OrganizationStatuses
set GunFreeSchoolsActReportingStatusCode = 'YesReportingOffenses'
where GunFreeSchoolsActReportingStatusCode = 'YESWITHREP'

update RDS.DimK12OrganizationStatuses
set GunFreeSchoolsActReportingStatusCode = 'YesNoReportedOffenses'
where GunFreeSchoolsActReportingStatusCode = 'YESWOREP'

Create new table to link People and Addresses

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Deborah Donovan
Authoring Organization(s)
Mississippi Department of Education
Email Address
[email protected]
Use Case Title
Create new table to link People and Addresses

Use Case Overview
We would like to be able to link a person (student and staff in particular) with all of their associated addresses, including mailing and physical addresses. There is not currently a way to link this data together for all use cases using existing tables.

Pull Request Number(s) (If applicable)
N/A
Use Case Background
See overview above. For example, we want to display the mailing and physical addresses for all students. Currently, there is no link available.

Components Affected
Delete everything from the list below except the components affected:

CEDS Data Warehouse (complete DW Design Overview Table)

CEDS Data Warehouse Parquet (complete DW Parquet Design Overview Table)


DATA WAREHOUSE DESIGN OVERVIEW (Includes Dim, Fact, and Report tables)


Add GradeLevelId to FactK12StudentAttendanceRates

This is for capturing needs not currently supported by the CEDS model. Please do not send or share actual data as examples in this issue or in attachments.

Author(s)
Johnny Cunningham
Authoring Organization(s)
CEDS
Email address
[email protected]
Use Case Title
Provide a concise description that defines the use case.
Add GradeLevelId to FactK12StudentAttendanceRates
Use Case Description
List the proposed element name(s), definition(s), and/or option set(s).

Use Case Background
Provide information related to why these changes/additions are needed.

Location of Element in the Domain Entity Schema

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.