Synchronizing Data with PASI

One of the primary techniques used when Integrating with PASI is to synchronize the data between PASI and another software solution.

Services

Architectural Considerstaions

Integrated solutions may fully synchronize its data with PASI. Synchronization is accomplished using the Is Data Available service. Through Is Data Available, PASI supports full synchronization of the following data:

The following questions should be considered when Synchronizing Data with PASI for each type of information being synchronized.

With respect to school year specific data, what school years will be synchronized with the solution?

The synchronization of the following entities is based on school year:

These entities can be synchronized for up to four school years. The calculation of which school years PASI is currently supporting synchronization is based on the current calendar year. Use the following to calculate the school years for which this information can be fully synchronized.

Sample Dates Calendar Year Calculation Calculated School Year Can Be Synchronized?

September 1, 2018
December 31, 2018

2018 2018 + 2 2020 ⇒ 2019/2020 No - Too far in the future
2018 + 1 2019 ⇒ 2018/2019 Yes
2018 - 0 2018 ⇒ 2017/2018 Yes
2018 - 1 2017 ⇒ 2016/2017 Yes
2018 - 2 2016 ⇒ 2015/2016 Yes
2018 - 3 2015 ⇒ 2014/2015 No - Historical School Year

January 1, 2019
June 30, 2019
August 31, 2019

2019 2019 + 2 2021 ⇒ 2020/2021 No - Too far in the future
2019 + 1 2020 ⇒ 2019/2020 Yes
2019 - 0 2019 ⇒ 2018/2019 Yes
2019 - 1 2018 ⇒ 2017/2018 Yes
2019 - 2 2017 ⇒ 2016/2017 Yes
2019 - 3 2016 ⇒ 2015/2016 No - Historical School Year

It is expected that data will most often change during the synchronized school years. While data can change in other school years, it is not expected to be as common.

  • Older school years (Current Calendar Year - 3 or more) are considered historical school years.
  • Future school years (Current Calendar Year + 2 or more) are considered future school years that are not yet accepted within PASI.

How will PASI data be captured locally within the integrated solution?

While each software solution will have its own data, when a solution sychronizes data with PASI it is expected to retain a local copy of the PASI data. This will:

  • Ensure that the software has access to the latest PASI information, even if PASI is not available
  • Provide a baseline for comparing PASI data against the data within the software for the purposes of identifying discrepancies.
  • To support confirmation of synchronization via Is Data Available hash calculations.

Even if the software solution does not use / manage / expose all of the data available from a synchronized PASI object, the software solution is expected to capture the entire synchronized object within this local copy of the PASI data. This will:

  • Allow the solution to display extra information to its users
  • Populate 'missing' data elements when submitting updates to PASI, thus avoiding loss of PASI data.

How will the solution identify differences between the data in PASI and the data within the integrated solution?

Changes to data in PASI may be made directly in PASI or they may original from an integrated software solution. As data is updated, it is important that the update is made available to all organizations that are synchronizing that data with PASI.

The Is Data Available service, provides the mechanism to notify integrated solutions about updates to data within PASI (regardless of where the source of the update may have originated). As part of data synchronization, an integrated solution is expected to compare the data from PASI with its own data to determine if the data is 'in-sync'.

Depending on the business needs, PASI recommends one of three approaches be used:

  • Notify users of the discrepancies and let users manually decide how to resolve the discrepancy. Typically this will result in the user:
    • Accepting the data from PASI, and the software updating its data accordingly.
    • Rejecting the data from PASI, and the software updating the PASI data to match the data within the software.
    • Manually updating the data in the software and the software making the same updates to the data within PASI.
  • Automatically accept the data from PASI, and notify the users that the data has been updated.
    • This approach usually applies to data that is typically updated outside of the software, such as diploma exam registrations.
  • Automatically accept the data from PASI without notifying the users of the software.
    • This approach usually applies to data that cannot be updated within the software, such as the definition of a school, a school authority, or a diploma exam.

What is the approach to using the Is Data Available Hash parameter?

Edge conditions or technical problems may result in the local copy of the PASI data being out of sync with the data within PASI, without Is Data Available notifying the software of the data change. The Is Data Available service supports the use of a Hash parameter allowing integrated solutions to validate it is synchronizing on all records accessible to the organization.

When the Is Data Available Hash parameter is used, the provided value is compared against a Hash value calculated by PASI. When these values does not match, Is Data Available will return a list of all records currently accessible to the organization. This list can then be used by the software solution to identify the records:

  • that can be synchronized with PASI and thus which must be included in the Hash calculation, and
  • that are out of sync and need to be updated in the local copy of the PASI data.

Students enrolling at a different school in a different school authority is an example of when a solution may be attempting to synchronize a record that is no longer accessible (as the student is no longer associated to the original school). Regular use of Is Data Available without a Hash value will not notify the integrated solution of the lost of student association, and therefore the loss of access to the student's information.

  • A student will remain associated to a school/school authority even if the student is no longer attending the school, provided the student has not enrolled at another school in a different/school authority. During this time, the software solution will continue to synchronize the student data.
  • Once that student enrolls at a different school in a different school authority, the original school/school authority will lose its association to the student. Is Data Available without a Hash will not notify the original school/school authority of the lost association.
  • The original school/school authority will become aware of the lost student association when it calls Is Data Available using a Hash that whose calculation included the lost student. The Hash value from the software solution will not match the PASI calculated Hash.
  • When the Hash does not match Is Data Available will return a full list of students accessible by the original school/school authority. Using that list, the integrated solution can then mark that student as a non-synchronizable student, exclude it from future Hash calculation, and no longer attempt to submit any student personal updates for that student to PASI. Subsequent Is Data Available calls with a Hash will then match.

Note in the above scenario, if the student does not show up for classes at the new/different school/school authority, that school/school authority will delete the student’s school enrolment. Immediately upon deletion of the student school enrolment, ownership of the student will revert back to the last school that has a valid Student School Enrolment for the student. When this occurs the integrated solutions for the last school will not be notified of the gained student association when calling Is Data Available without a Hash. It will be through the use of Is Data Available with a Hash that the integrated solution will be made aware of the gained student association.

A Hash match failure is intended to provide notification to the integrated solution of cases where access to a record has changed as a result of factors that are external to the software solution or school authority. Under normal usage hash match failures are expected to be rare. Hash match failures can also, however, occur if there is a defect or race condition in the integrated solution's synchronization algorithm. Frequent and persistent hash match failures usually indicate that there is a defect in the integrated solutions synchronization algorithm that must be addressed.

An integrated solution must call Is Data Available with Hash parameters daily to validate it is synchronizing the correct entities and thus is in sync with PASI. PASI Is Data Available processing using a Hash is significant thus an integrated solution must only use the Hash parameter at most once per hour to avoid excessive load on PASI.

How will your integrated solution resume synchronization after PASI connectivity has been lost and subsequently restored?

For various reasons, an integrated solution may occasionally lose connectivity with PASI and thus synchronization/Is Data Available usage will be interrupted.

When connectivity with PASI is reestablished, the integrated solution's synchronization process must have the ability to resume Is Data Available calls from last Maximum PASI Core Version that has been successfully synchronized.

For example if the last successful call to Is Data Available returned a maximum PASI core version of 1000056 and connectivity with PASI is lost, then the first call to Is Data Available after connectivity with PASI is reestablished must use 1000056 as the Maximum PASI Core Version. This will return all relevant PASI updates that occurred while connectivity to PASI was lost.

How will the integrated solution handle unexpected version numbers returned from Is Data Available?

When PASI processes a data update, it will immediately return a new PASI version number. These PASI data updates initially flow into the PASI database and then into the PASI Active Cache. Depending on PASI’s load, the flow of updated PASI data into the PASI Active Cache may be slightly delayed. As a result it may be possible for an integrated solution to submit an update to PASI, get a new version number, and then have Is Data Available or a Get service return a lower version number.

An integrated solution’s synchronization must utilize the Get service ‘Expected Version’ parameter when processing results from Is Data Available. The ‘Expected Version’ parameter must be set to the value returned by the preceding Is Data Available service call. When used in this manner, the Get service will only return ‘expected’ results and thus the integrated solution will not have to implement and maintain special ‘unexpected’ version logic.

How will the integrated solution use the ‘Get’ service functionality for the retrieval of multiple records with one service call?

A number of PASI ‘Get’ services allow a list of IDs to be specified thus returning multiple records using a single service call.

To avoid excessive and unnecessary PASI serviced calls, an integrated solution must make use of the ID ‘list’ when IDs of several records are known and must be retrieved. This is particularly useful during synchronization when Is Data Available returns IDs of several records that have changed.

What is the typical latency between changes occurring in PASI and those changes being available to your users?

An integrated solution's synchronization process must be an asynchronous process that is continually executing using the Is Data Available ‘long polling’ approach. This will ensure the solution is immediately made aware of changes to data.

For example if a student is attending two schools at the same time, and only informs the first school of their address change. The second school will be notified of the address change immediately after the first school submits the change to PASI, because it is using the Is Data Available long polling approach.

What approach is used to handle deleted records?

PASI will notify integrated solutions about deleted data via the associated Get service, typically using the Is Deleted attribute.

Also, a solution may no longer have access to a particular record, typically as a result of a loss of association to the student. When this occurs, the the hash check will fail. During the resynchronization of data, records that the integrated solution no longer has access to can be identified as they are no longer returned as part of the Is Data Available response.

It is expected that once an integrated solution learns of a deleted record (or a record they no longer have access to), that record is removed from the solutions local copy of PASI data, and if needed the Reference ID and the PASI Core Version are removed from the solutions data.