An Open Letter to FracFocus
FracFocus.org is the preferred chemical disclosure registry for the oil and gas (O&G) industry, and use of your website by the industry is mandated by some states and regulatory agencies. As such, we hope you’ll be responsive to this call by FracTracker, other organizations, and concerned citizens across the country to live up to the standards of accessibility and transparency required by similar data registries.
A Focus on Data Transparency
Recent technological advances in high volume hydraulic fracturing operations have changed the landscape of O&G drilling in the United States. As residents adjust to the presence of large-scale industrial sites appearing in their communities, the public’s thirst for knowledge about what is going on is both understandable and reasonable. The creation of FracFocus was a critical first step down the pathway to government and industry transparency, allowing for some residents to learn about the chemicals being used in their immediate vicinities. The journey, however, is not yet complete.
Design Limitations on FracFocus
Query by Date
Even with the recently added search features there is no way to query reports by date. Currently a visitor would be unable to search by the date hydraulic fracturing / stimulation was performed, or when the report itself was submitted. Reports can only be viewed one PDF at a time, which would take someone quite a while to view all 68,000+ well sites in your system.
Aggregate Data Downloads
In October 2013, you informed us that “each registered state regulatory agency has access to the xml files for their state but they are not distributable from FracFocus to the public.” We must ask the reasonable question of “why not?” We understand that setting up a downloadable data system is a time-intensive process, as we manage one ourselves, but the benefits of providing such a service more than compensate for the effort expended. It is no longer possible to aggregate data, either automatically or manually, because of bandwidth limitations that keep users from downloading more than an arbitrarily limited number of reports in a single session. Considering public concern over the composition of frac fluid, as well as the volume and geographic extent of complaints of drinking water complaints to be related to O&G extraction, prudence would suggest making the data as accessible as possible. For example, making the aggregated data available to the public as a machine-readable download would greatly reduce the load on your servers, because users would no longer be forced to download the individual PDF reports. Changes in the way the reports are curated would also improve efficiency and reduce your server load; we would be more than happy to discuss these changes with you.
An Issue of Money?
The basic infrastructure to provide this service via FracFocus.org is already in place. An organization like the Groundwater Protection Council with a website serving some of the world’s wealthiest corporations loses credibility when making claims that “we have no way to meet your needs for the data.” Withholding data from the public only serves to compound the distrust that many people have with regards to the oil and gas extraction industry. Additionally, agencies that use FracFocus as a means of satisfying open government requirements are currently being short changed by the lack of access to your aggregated datasets; restricting access to data that is in the public interest is fundamentally at odds with data transparency initiatives, including the President’s 2013 Executive Order on Open Data.
One Small Step for a Company…
Within this discussion is a simple realization: The Ground Water Protection Council, Interstate Oil and Gas Compact Commission, participating companies and states, and the federal government should recognize that data transparency is not merely a lofty ideal, but an actual obligation to our open society. Once that realization has been made, the path of least resistance becomes clear: you, FracFocus, should make all of your aggregate data available to the public, beginning with the easiest step: the statewide datasets that are already being provided to government agencies.
FracTracker operates in the public interest. We – and the thousands of individuals and organizations who use our services and yours – request no less from you. Thank you for addressing these critical matters.
Sincerely,
-The FracTracker Alliance-
On a related note, check out the May 15th letter written to the Bureau of Land Management by our partners in data transparency, SkyTruth:
SkyTruth Issues Letter to BLM Director on Public Disclosure of Fracking Chemicals
Despite Mr. Everley’s claim (in a recent piece in Energy Wire 5/2/14 http://www.eenews.net/login?r=%2Fenergywire%2Fstories%2F1059998864%2Fsearch%3Fkeyword%3DFracFocus) that FracTracker “pushed (our open letter) out to the media instead of contacting FracFocus directly,” the open letter was written and posted only after extensive inquiries with FracFocus –both recently and on prior occasions – resulted in no viable solution/response. FracTracker has saved this correspondence and it can be read below. Readers can reach their own conclusions. – Brook Lenker, Executive Director, FracTracker Alliance
Question #1:
Date sent: March 27, 2014
Date reply received: March 27, 2014
Question: In order to allow us to efficiently update our databases while greatly reducing demand on your servers, are you willing to make the submitted data available to us as a downloadable .csv file that is regularly updated (monthly or more frequent) as new or revised disclosure reports are uploaded to the FracFocus system?
Response: Mike Nickolaus – Mr. Lenker: Unfortunately we are unable to comply with your request for downloadable data at this time. However, the topic of data access is under discussion so as we move forward this may become a possibility. Please keep checking back every month or two and I will keep you apprised of our situation.
Question #2:
Date sent: March 28, 2014
Date reply received: March 28, 2014
Question: FracFocus 2.0 is collecting information via machine-readable XML. In order to allow us to efficiently update our databases while greatly reducing demand on your servers, are you willing to provide us access to all the submitted XML files via an FTP server?
Response: Mike Nickolaus – Brook: I’m sorry but we can’t do that either. Each registered state regulatory agency has access to the xml files for their state but they are not distributable from FracFocus to the public.
Question #3:
Date sent: March 31, 2014
Date reply received: March 31, 2014
Question: So how about this – If the machine-readable data is only accessible to participating government agencies, what about the PDFs?
Are you willing to provide us access to an FTP server allowing us to download all of the PDF chemical disclosure reports currently in the FracFocus system, and to periodically download all new or revised PDF reports as they are added to the system in the future? This would still allow us to efficiently update our databases while greatly reducing demand on your servers.
Response: Mike Nickolaus – Brook: I appreciate the fact that you are trying to offer alternate solutions. It is just at this time that we have no way to meet your needs for the data. As I mentioned previously, we are working on possible solutions including a pdf bundler application but at present these are not available so the files can only be accessed one pdf at a time. Please feel free to check back with us in a couple of months to see if we have a more flexible means of information access. My apologies for the inconvenience and thanks for using FracFocus.
Question #4:
Date sent: April 1, 2014
Date reply received: April 1, 2014
Question: Mike – thanks for bearing with me. This is the last question about features and it doesn’t involve a download.
In order for a user to identify new reports on the site, we have to check every single report in the FracFocus system. This is highly inefficient and places unnecessary load on your servers. Can you provide us with an RSS feed (or similar machine-readable publishing standard) with basic metadata (at a minimum, API number and date) for all new and revised reports uploaded to FracFocus?
Response: Mike Nickolaus – Brook: I’m sorry but we don’t have that kind of capability. However, the search function let’s you search by several different dates including before, between and after certain dates. This will return a list which can then be sorted by date.
Just to let you know, using the data extracted from FracFocus.org by SkyTruth.org, I’ve written several programs suitable for public consumption that tie FracFocus data with chemical toxicities and earthquake locations. I am now in the process of open-sourcing those programs in anticipation of FracFocus’s acquiescence to bulk data download. The program’s modalities vary from traditional web server based, to embedded web server based, to desktop based. The data is held in JDBC-compliant databases and/or web-based Apache Solr indexes and/or local Apache Lucene indexes.
In short, any way it’s convenient, it’s pretty much ready to go.
I’m just waiting for the data download’s to become available.