USAID Open Data Policy: ADS 579

USAID Open Data Policy: ADS 579

Because the USAID ADS 579 policy has serious implications for the work of M&E and other offices that do surveying, create beneficiary databases, carry out research or collect datasets for any monitoring or analytical purposes, it’s important for M&E and program staff to understand what’s required.

Last Spring we brought to your attention the New USAID open data policy (ADS 579) that went into effect on Oct 1, 2014.  All agreements negotiated or amended after October 1, 2014 must comply.  Per 579.3.2.1 “USAID staff, as well as contractors and recipients of USAID assistance awards (e.g. grants and cooperative agreements), must submit any Dataset created or collected with USAID funding to the Development Data Library (DDL) in accordance with the terms and conditions of their awards.”  According to the policy, “if the partner has an underlying, machine-readable dataset that contains data funded by USAID, the partner must submit the dataset to the DDL.”  Because the USAID ADS 579 policy has serious implications for the work of M&E and other offices that do surveying, create beneficiary databases, carry out research or collect datasets for any monitoring or analytical purposes, it’s important for M&E and program staff to understand what’s required. 

This Message is to Provide Guidance on the Following:

  1. Why Did USAID Develop ADS 579?
  2. Do all U.S. Government Agencies Have Similar Requirements?
  3. How Does Data Get Submitted?
  4. Why is the International Development Community Concerned about ADS 579?
  5. Where do things Stand Now on ADS 579?

Why Did USAID Develop ADS 579?
The policy was developed in response to Obama’s U.S. Executive Order 13642 on “Making Open and Machine Readable the New Default for Government Information.”  USAID developed ADS 579 to instruct its staff on compliance.  The intent of the policy is to make more data from USAID programs publically available, open and readable for access by researchers, international development professionals and others who may be interested.  See USAID Open Data Set Policy Frequently Asked Questions.

Do all U.S. Government Agencies Have Similar Requirements:
Note that given ADS 579 is based on a U.S. Executive Order, it’s reasonable to assume that at some point all U.S. government agencies will adopt policies similar to USAID’s, though this has not yet occurred. For example, we have not received any notification from the U.S. Department of State or the U.S. Department of Agriculture about making our datasets publically available. 

How Does Data Get Submitted?
Currently there is no place to upload a completed dataset on USAID’s website.  Rather, the implementing organization must complete a Data Submission Form, after which the organization will be sent further instructions from USAID. 

Please inform the Monitoring, Evaluation and Learning Unit of any datasets you submit to USAID so we track what’s submitted and ensure Global Communities is in compliance with this rule.

Why is the International Development Community Concerned about ADS 579?
Many concerns have been raised within the development community about possible ethical, policy and compliance issues raised by ADS 579, including whether sharing of datasets on our beneficiaries could put anyone at risk.  There is also concern that the policy has been rolled out too quickly before more thought was given to ensuring it can be implemented soundly and provide data useful to those who may want to access it.  Interaction, the consortium organization representing U.S. international development NGOs in Washington, has been leading this effort and prepared this summary of concerns and recommendations in the Spring of 2015: See Interaction Summary of Concerns on USAID Open Data Policy.

Key questions raised to USAID by Interaction Include: 

  • Will the public release of data influence individuals’ responses or willingness to consent?
  • Do organizations now have an ethical obligation to obtain informed consent for all data collected from individuals, including process and monitoring?
  • Will all data collection activities involving individuals be subject to Institutional Review Board (IRB) approval?
  • If organizations are asked to submit datasets collected before the policy’s implementation, do they have an ethical obligation to go back to the individuals to re-obtain consent, given that the stipulation that the data that would be made public is unlikely to have been part of the original consent process.

Interaction requested, but was not successful in obtaining from USAID, a moratorium on implementation of ADS 579, and creation of an advisory group to examine the best way to carry out the policy.  As a result, all USAID-funded contracts, cooperative agreements and grants negotiated or amended since October 1, 2014 must comply.

Where do Things Stand Now on ADS 579?
Interaction requested, but was not successful in obtaining from USAID, a moratorium on implementation of ADS 579, and creation of an advisory group to examine the best way to carry out the policy.  As a result, all USAID-funded contracts, cooperative agreements and grants negotiated or amended since October 1, 2014 must comply.  Please review your agreements carefully as well as the ADS 579 guidance provided to ensure you’re submitting what’s required.

In the interim, Interaction is still discussing the issue with USAID and shared this information recently relating to (1) the timing for the rule-making process; (2) some of the changes that are being made to the policy; and (3) USAID’s plans for developing best practice documents.

1.  USAID Must Get Public Approval for New 579 Regulation: To comply with USG rules on development of new regulations, USAID was expected to complete its internal clearance process in late August and then send its proposal to “OMB and the Federal Register for a review period of approximately 90 days.  If no comments or revisions are received from the OMB / Federal Register review, it will then be posted for public comment. You may wish to review this site for more details on the overarching process.”

2.  Expected Changes to the Policy: Based on Interaction’s suggested revisions, Interaction was advised that the following changes in ADS 579 should be expected:

  • Exemptions: The new version of the policy will include exemptions to data submission, although these will be few. Interaction has no idea what exemptions may be.
  • Informed consent: Organizations will need to submit a copy of the informed consent form used in data collection (not signed copies – just the form itself).
  • Timing of submissions: The revised policy will provide more guidance on the timing of submission of datasets. For monitoring data (or data that is regularly updated), the policy will call for data to be submitted on at least an annual basis, or as frequently as determined by the AOR/COR.
  • Retroactive application of the policy: There is no change here. The policy will still apply retroactively, although Interaction has heard that some AORs/CORs have been granting partners with exemptions, especially for work taking place in sensitive countries.
  • Notifications of data access levels: Implementing partners will be notified about the access level set for a dataset, and of any changes made to the access level.
  • Redress regarding data access levels: The revised policy itself will not address what is to be done when USAID and an implementing partner disagree about the access level selected for a dataset. However, USAID indicated that it would add this to ADS 579.  In these situations, USAID’s General Counsel may be responsible for making the final call.
  • Third party rights to data: It will still be up to implementing partners to get third party rights to data. That is, if part of the data being submitted “belongs” to another organization or partner country government, the implementing partner – not USAID – will need to get permission to submit that data to the DDL.

3.   Best Practice Documents: USAID plans to develop several best practice documents to provide additional guidance for implementation. The topics under consideration are below. If there are other topics you think USAID should address, please contact Susan Morawetz, Senior M&E Specialist, who will forward the suggestion to Interaction.   USAID would like to develop these documents in collaboration with implementing partners and others and plans to finalize these documents by Spring 2016. Topics will likely include:

  • De-identification and anonymization (once finalized, implementing partners would need to abide by this guidance)
  • Informed consent (may be addressed as part of the topic above)
  • Creating a data management plan
  • Codebooks and data dictionaries
  • Step-by-step guide to implementing the open data policy