How San Francisco purged scores of records requests that were available to the public
By Joe Dworetzky ✹ Bay City News
PART II
In late February 2025, a Bay City News reporter and another frequent user of the NextRequest system began to ask for details about the visibility settings on the system and why some departments like the Department of Homelessness and Supportive Housing (HSH) kept all their prior requests invisible.
A few weeks later, on a routine visit to the search portal, the reporter observed that the visible records in the system were only 13,297; a week before, there were close to 40,000. Tens of thousands of requests and hundreds of thousands of related documents associated with those requests had been purged from the search portal and were no longer visible for public search. Despite his close observation of the system, BCN’s reporter had no idea that a purge was coming.
After inquiry, the Department of Technology (DT) said the requests and related documents were removed from the system on May 1 pursuant to a newly adopted “data retention policy.” It said the missing requests were “permanently removed” and “cannot be recovered.”
When asked when the policy was created, DT produced a draft of a memorandum dated March 21, that was to advise city departments of an “Upcoming Data Retention Policy Implementation.”
The memo announced that a new data retention policy for the NextRequest system had been adopted and would be implemented beginning on April 1. Under the new policy, each month going forward, DT would send an email to the departmental administrators on the 15th of the month, “listing the requests and documents scheduled for deletion the following month.” That would provide the departments “time to review and, if necessary, exempt data before the next purge cycle.”
Unexempted records and related documents would then be purged automatically on the first day of the following month, beginning May 1.
On April 1, an email announcing the new policy went out to the departments using the NextRequest system from the “DT NextRequest Product Management Team.” (Subsequent inquiry unearthed the fact that the Product Management Team only had one member — Arlene Licudine-Barker — a Principal Business Analyst for the DT.)
While some of the city’s information technology policies are set by the Board of Supervisors, in light of the specialized and technical nature of the material, many are made by the Committee on Information Technology (COIT).
COIT holds regular public meetings. It includes 16 voting and two nonvoting members from different parts of city government. The mayor, the president of the Board of Supervisors, the controller, the city administrator and the chief information officer are all permanent members. The committee maintains a public website that shows its meeting schedule and agendas for its meetings.
The data retention policy was not approved by COIT.
Matt Yankee, chair of the task force, said the DT did not communicate with the task force about the purge and only learned about it after the fact.
San Francisco’s Administrative Code has provisions that governs retention and destruction of city records. Section 8.3 allows records less than five years to be deleted only if the deletion meets a series of criteria including that 1) their destruction “will not be detrimental to the City and County or defeat any public purpose;” and 2) if a “definitive description” of such records and the retention period are set forth in a schedule that is approved by the mayor or the mayor’s designee.
DT did not obtain the approval of the mayor for the purge of records and related documents. There is also no record of consideration of whether the purge would defeat the public purposes of the Sunshine Ordinance.
According to DT, no approval of the “data retention policy” nor the purge was necessary because this was a “system configuration” not a citywide policy. DT’s position is that it could eviscerate the contents of the search portal solely through its own personnel with input from the departments using the system.
So much for the Sunshine Ordinance’s declaration that city departments “exist to conduct the people’s business. The people do not cede to these entities the right to decide what the people should know about the operations of local government.”
✹
DepartmentS Reviewed Requests before Purge
DT’s policy said that departments would receive an email notification on April 15 and the 15th of each following month “listing the requests and documents scheduled for deletion” so the departments would have “time to review and, if necessary, exempt data before the next purge cycle.”
On April 1, DT sent an email to departmental administrators saying, “a report to list records submitted for deletion will be sent to all department administrators every 15th of each month.”
BCN filed a public records request with DT that asked for the April 15 report referenced in the email.
DT sent BCN a 985-page document that said at the top “These requests and documents will be deleted on May 1, 2025. You can mark them as exempt if you don’t want them deleted.”
When BCN analyzed the report, it discovered that it only included records for the 10-month period from 2017 up through Aug. 16, 2018, although under the policy it should have included all search requests and documents thorough the April 1, 2022, cutoff date.
When BCN asked DT to comment on its use of a partial list of the requests and documents teed up for the purge, DT said that the report it provided to BCN was just a “sample report” that was not released to departments. “Instead, each department was instructed to produce their own report and review their data records.”
By its own account, DT did not follow its stated policy. Perhaps that was rendered inconsequential by departments producing their own reports and reviewing the materials to be purged directly on the NextRequest system. While BCN did not resolve that issue, there is some support for the department’s position.
The most recent post purge system report shows that there are 487 records requests still on the system that are within the purge’s date range, suggesting that those requests were exempted from the purge. The percentage is small — 487 out of more than 53,000 requests in the purge — but not insignificant. Moreover, while most of the exempted requests were from the Department of Public Works (210) and the Department of Health (206), several other departments also exempted requests from the purge, suggesting that there was at least some pre-purge review in a number of departments.
While BCN was unable to reach a conclusion on whether DT’s failure to follow its stated policy was made harmless by its instruction that departments “produce their own report and review their data records,” it is clear that the purge drastically reduced what was available to the public when BCN started looking into the issues.
DT estimates that it purged 53,966 requests but said it did not know how many documents associated with those requests were purged.
BCN estimates that as many as a million documents were purged, based on the ratio between requests and documents in the “sample report” DT produced. DT disputes that estimate but is apparently unable to determine the actual number, a telling problem with the process it followed.
✹
What was purged
BCN’s investigation did not find evidence that DT implemented the policy or conducted the purge to hide any specific documents or hinder any investigation, but because of the way that the city allows departments to keep requests invisible and the way DT handled the purge process, there may be no way to determine all that has been lost as a result of the purge.
Much of it was surely inconsequential, but there is reason to think that some of the purged requests and documents were of importance.
Working from a pre-purge system report, BCN found many examples of requests that probed for information about corruption or mismanagement in city government.
BCN turned up more than a hundred records requests from before the April 1, 2022, cutoff date that called for city departments to produce public records concerning Mohamed Nuru, the now convicted head of DPW, and his relationship with his associates — many also who were later convicted or the subject of a plea arrangement.
Many of the requests asked for emails between Nuru and his web of associates throughout the city. Others asked for personal calendars. There were also a dozen requests asking about the San Francisco Parks Alliance, the nonprofit currently subject to a city funding freeze as well as civil and criminal investigations by the City Attorney and the District Attorney.
LEARN MORE: A guide to making a public records request in San Francisco
Cross-checking the lists shows that only about half of the requests that BCN identified were exempted from the purge.
BCN’s investigation did not attempt to check the tens of thousands of purged requests for all areas that might be of future importance, but one of the BCN searches, ironically, offers an illustration of how the purge has gotten rid of things that might well come in handy in the future.
BCN found dozens of deleted requests lodged by an anonymous requester using the name “Anonymoose.” (The Sunshine Ordinance and state law allow requesters to ask for public records anonymously to avoid the risk of retaliation.)
From reading his records requests, it is clear that Anonymoose was a gadfly and raised many issues relating to the city’s non-compliance with the Sunshine Ordinance. In more than 20 of his deleted requests, he presented — with spice and suspicion — his demands that the city live up to the requirements of the ordinance.
One of his concerns was that instead of helping requestors get the materials they were trying to find (as the Sunshine Ordinance requires), department administrators were communicated among themselves on strategies to avoid or delay producing inconvenient of damaging records.
In record request 22-861, filed Feb 9, 2022, Anonymoose wrote that he had found a thread relating to the city’s shared spaces initiative in which the Planning Commission allegedly directed Public Works to ‘hold off responding further to the request’ … due to the then-politically sensitive status of the shared spaces proposals …”
As evidence, he identified a document as “Document 11058463” that he said contained the exchange. In the system report, the document is hyperlinked, but when BCN tried to follow the link, it only led to a DT notice that advised that requests and documents older than 3 years are deleted.
The notice explained that the “secure and irreversible removal of unnecessary data from our systems ensures compliance with our data retention guideline and enhances our data security practices.”
The notice does not provide the contents of Document 110584463, but it closes politely saying, “Thank you for your contribution to our data management efforts.”
BCN’s efforts to contact Anonymoose have not been successful, and it has not been able to bring Document 110584463 out into the sunshine. One wonders what else was lost in the purge.
