The Dream of Online Access to USDA Operations

In 1992 we had the dream of permitting farmers online access to ASCS, SCS, Farmers Home applications.  In the initial Infoshare pilot we found very limited adoption.  As I've observed from a distance the different embodiments of that dream over the years, I've always been curious how many farmers were really getting online and making use of the capabilities USDA provided. But despite my suggestions over the years, I'm not aware of any Federal site which publishes usage figures, so there's no way for a member of the public to see whether progress is being made.

Recently I found a clue, at least for FSA/NRCS/RD, thanks to the requirement for public notice on data requirements.  (The first time in my life I've really seen a value for that procedural requirement.)

Here is the Federal Register document from USDA on the information collection requirement for e-Auth.

"The USDA eAuthentication Service provides public and government businesses single sign-on capability for USDA applications, management of user credentials, and verification of identify, authorization, and electronic signatures. USDA eAuthentication obtains customer information through an electronic self-registration process provided through the eAuthentication Web site. The voluntary online self-registration process applies to USDA Agency customers, as well as employees who request access to protected USDA web applications and services via the Internet. Users can register directly from the eAuthentication Web site located at www.eauth.egov.usda.gov. The information collected through the online self-registration process will be used to provide an eAuthentication account that will enable the electronic authentication of users. The users will then have access to authorized resources without needing to reauthenticate within the context of a single Internet session."

 "Description of Respondents: Farms; Individuals or Households; Business or other for-profit; Not-for-profit institutions; Federal government; State, Local or Tribal Government.
Number of Respondents: 114,256.

 There's no breakdown given for how many of the respondents are actually farmers.  My guess would be about 80,000 to 100,000, which might be from 10 to 25 percent of potential users.

More Action on MIDAS

FSA has new information on MIDAS and new releases of software.  See here.

I'm getting ready for a trip over the weekend so haven't looked at the information.  Possibly the project has its own momentum and logic. 

ARRA and MIDAS

This piece on the ending of the Recovery Act database reminded me--MIDAS got $50 million if I remember correctly.  Maybe not, maybe the $50 mill was partly to upgrade the creaky technology at the time.

I do wish they'd included some usage figures on the website--how much did the media and others actually use the site?  I know while I checked it a few times early on, I never did go back to see what if any updates for MIDAS had been added.  It may well be the best contribution of the effort was to establish a precedent, to teach people what was involved so the next try can be more useful.

House Ag Appropriations

From the House appropriations ag subcommittee;

Here's the committee report on MIDAS:


"Information Technology Waste

.—GAO and USDA’s OIG have issued reports that highlight poor program performance in the past and uncertainty regarding USDA’s capacity to effectively manage IT acquisitions in the future. Auditors found that the Secretary halted further development on the MIDAS program after spending almost $500 million for nearly a decade on planning and development of this critical system. This investment of time and limited resources has resulted in the delivery of about one-fifth of the functionality intended for twice the projected cost. While the Secretary has highlighted saving hundreds of millions of dollars on IT, the Committee notes that MIDAS is a prime example of government waste and inefficiency. MIDAS is still expected to cost another $330 million over the lifecycle of the project, yet the system will have severely reduced capacity. The total cost will equal almost three times the original projections.

GAO noted that problems with MIDAS were due to the lack of implementation of USDA and Farm Service Agency (FSA) program management policies and best practices covering key disciplines such as requirements for development and management, project planning and monitoring, system testing, and executive-level governance. Following project stoppage, the Department has been exploring other options—at an additional cost to taxpayers and time spent on these modernization efforts—to provide the functionality that USDA had promised Congress and the agricultural community, including a modernized acreage reporting system and an online office for American farmers and ranchers to access. Given the lack of IT leadership demonstrated by the Secretary on the MIDAS investment, the Committee remains concerned as to whether the Department will be any more successful with IT acquisition activities moving forward than it was in the past with MIDAS. The Committee includes statutory language that places spending controls on both MIDAS and other IT acquisitions.

FSA IT.


—FSA’s management of certain IT projects has produced increased costs, bloated budgets, and inaccurate budget estimates. These projects include the MIDAS program and increased or inaccurate charges from the National Information Technology Center, for which costs have tripled since fiscal year 2014. The agreement includes statutory language that allows FSA to release funds for farm program delivery IT projects only after review by the GAO and approval by the Committees on Appropriations of the House and Senate. The roadmap submitted by FSA in fiscal year 2015 was the first step to bringing accountability and guidance to almost a decade of mismanagement. In this regard, the GAO and the OIG are recommending that FSA establish a plan to guide the agency in adopting recognized best practices and in following agency policy. The GAO also recommends that the agency adhere to specific practices within key management disciplines before proceeding with further system development. FSA is directed to continue quarterly briefings in writing for the Committees on Appropriations of the House and Senate regarding all IT projects and activities related to farm program delivery.

And an "attaboy" for NRCS:

The Committee commends NRCS for its organizational realignment of administrative functions and appreciates the savings this will generate. NRCS has worked to become a more efficient, accountable organization, and the Committee encourages NRCS to work with other agencies within USDA to do the same

GAO Report on MIDAS

Full report

It seems a bit harder on USDA than the OIG report.

What's Wrong With the Auditors?

The old question, from the Roman poet Juvenal, is:  "quis custodiet ipsos custodes?"

Earlier I posted about the new OIG report on FSA's MIDAS project. I've lost track of all the GAO and OIG reports critical of ASCS/FSA/USDA's automation efforts.  Juvenal's question doesn't quite fit--nowadays it implies some misconduct while my point is directed towards effectiveness.

In other words, given all those audit reports you'd think there would be some improvement over the years, but USDA and its agencies still seem to be ineffective in doing large IT projects.  I wonder why?

Some possibilities:

• IT procurement and development of IT systems keeps getting more complicated, so the bureaucracy's learning curve as embodied in the GAO/OIG reports doesn't gain on the difficulty curve of the projects.
• the USDA bureaucracy is incapable of learning, maybe because the policy officials turnover too fast, there's no insitutional memory, there's lack of ability or training, or something else.
• the auditors give bad advice, either misleading the bureaucracy on how to correct the problems or misidentifying the problems
• Congress fails to do good oversight--using the reports to hold the bureaucrats feet to the fire, or maybe they focus on the wrong issues.
• Congress fails to provide the money to do well
• the President and OMB fail to follow through on the reports
• the IT projects conflict with an outdated orgnaizational structure and culture.

I suspect all of these issues may apply.

OIG on MIDAS

OIG released an audit report on MIDAS last month.

Hat tip to this post from the Capital Press which represents what the media might make of the report.

"The federal government put a man on the moon, but 46 years later it can’t come up with a computer system for the USDA Farm Service Agency.
Such is the plight of the federal government in the 21st century. When it comes to computers, Uncle Sam is — how should we say it — a few bauds short of being online.

May have more thoughts when I read the report.

Farewell to System/36?

From the NASCOE newsletter

While I am sure most of your NASCOE officers will be covering some of the highlight of the past Calendar Year I will be focusing on the NEA Programs side of NASCOE. This year we should finally say goodbye to the wretched beast some call the 36 or AS 400. Of course, this good bye brings something new(ish).
We just received MIDAS release 2.0 which we’re told will work without a single hiccup.
We will at last have one system of record for maintaining Producers name, address, and TIN.
We are also anticipating new MPP software, new MAL software, and as well new software for ARC/PLC, which is already out. We’re also now using different software for FSFL’s.

 Those were the days, my friends. I remember them well (cue Maurice Chevalier in Gigi, a stage production of which is now at the Kennedy Center).   I remember Chris Niedermeyer as "trail boss" in 1989? talking about running out of space on the System/36 and the need to immediately get new hardware.   And I'm pleased to see the "one system of record," only about 16 years late.

FSA IT Crimped

On page 29 and 30 of the Cromnibus, FSA IT is somewhat crimped: half the $132 mill is withheld pending a detailed analysis/report on projects over $25K.  (Copy and paste from GPO documents is unsatisfactory, so read yourself, if interested.  Everything has to fit the "Farm Service Agency Information Technology Roadmap", which sounds like something which should be available on the internet?

Lack of Compatibility

From NASCOE's June update.:

"We recently learned that OMB /DAFP made a visit a Pennsylvania county office a few weeks ago. The visit focused on the progress of MIDAS. The group was curious why MIDAS is not being used as the platform for the new farm bill programs, and asked why we continue to utilize 2 systems (Web Farm & MIDAS). We understand that the group seemed to be surprised at the lack of compatibility and interaction of our software programs. Reports were that the visit was beneficial, yet eye opening for the OMB/DAFO group." 

 One thing the IT types who were involved in Infoshare never understood, and I failed to make them understand, was the problem of transitioning from legacy systems to new systems.  We went through hell in 1985-7 when we transitioned to the System/36 and everyone was happy to forget the pains.  As Santayana said: “Those who cannot remember the past are condemned to repeat it.”

FSA's Budget

From the budget document

The 2015 Budget proposes a level of $1.45 billion. As part of the 2015 budget, FSA is developing a “Model Service Center” concept that will result in service centers that are better equipped, better staffed, and will provide improved service to customers. Part of the plan is to close or consolidate 250 offices and restructure the workforce to more effectively leverage its human capital. With reduced redundancies, streamlined business processes, and a reduced national footprint, FSA will be able to deliver programs more efficiently. In addition, FSA proposes additional staffing for farm loans in anticipation of increased loan demand. FSA is continuing to modernize its information technology (IT) systems and move away from unreliable, obsolete systems. Billions of dollars of annual farm program payments, conservation payments, and loans to producers have been dependent upon antiquated IT systems. FSA must continue to upgrade its IT infrastructure in order to provide more efficient and reliable services to producers. 

FSA’s MIDAS program is a critical part of its IT modernization efforts that supports farm program delivery with streamlined business processes and integrated applications that share information and resources efficiently. MIDAS achieved an initial operating capability release in April 2013 that modernized the storage and retrieval structure of current farm records and integrated this information with land use data, land imagery data and producer information. The system will permit FSA employees to access and better validate program eligibility data and financial services data from a single source and improve customer account management.

What's Happening on MIDAS: Congress Wants to Know

Getting around to the omnibus appropriations bill:  the first part of the report  dings USDA for failing to report timely on MIDAS.  Also says:

"In order to leverage existing capacity and expertise within the Department, the Secretary is directed to explore the creation of a Center of Excellence for loan servicing support functions in order to provide consolidated customer service, field office support, and centralized loan services to USDA agencies and other Federal agencies. The Secretary shall consult with employee representatives and management in the Farm Service Agency Farm Loan Information Technology, Accounting, and Finance Office loan servicing support functions; the Rural Development Deputy Chief Financial Officer and Deputy Chieflnformation Officer functions; and the Rural Housing Centralized Servicing Center."

Not sure what the quote means, except that someone, an employee, a contractor, or a congressperson has a bee in their bonnet.  "Center of Excellence"?  Sounds like bs to me.

ACA and FSA MIDAS

Being old, I've no need to sign up for Obamacare, so I've no personal experience with the website.  From what I've read, however, apparently the "navigators" who are helping people sign up are using the same software/website as those who are signing up on their own.  If so, that seems wise to me.  It's hard enough to keep one set of software operational and supporting the program.  It would be much harder to keep two sets up-to-date: one set for the public and one set for the government employees.  It would be particularly challenging when you have legislation passed late which requires changes to implement.

I don't know how MIDAS is set up, but I hope they've followed the same approach. 

Farm Bill Time Again

The House and Senate conferees will meet next week on the farm bill. The Rural Blog passes on speculation about possible effects on FSA offices.

I wonder whether FSA employees are comparing the rollout of MIDAS (which seems to have had problems, though not very visible outside the walls of FSA) with the rollout of ACA. 

Acreage Reports and Performance (27 Years Ago and MIDAS)

I noticed in recent NASCOE documents a brief discussion of "performance" problems, apparently due to MIDAS.  Today I see ND FSA quoted in this article on problems with acreage reporting due to MIDAS performance problems. 

I'm not a good person, so there's a smirk on my face, for which I apologize to everyone involved--I'm sure they are doing their best.

As some consolation I'd recall our problems with the ASCS-578 process when we first automated on the System-36.  The initial program design had one entry screen for each parameter for a field or subdivision (i.e., "corn" would be on one screen, "grain" or "silage" would be on another).  Do I need to add that with the first 36's we didn't move from one screen to the next very quickly?  The net result was something which was unusable, though with the combination of ignorance and rigidity too often found in the South Building we (I) earnestly explained to the state specialists that counties had to use the software.

Here my memory fades--I think we officially used the initial design for 1985, backed off to a data load process for 1986, and perhaps came up with a revised process for 1987, though maybe it was 1988.  The new process was an improvement, if I say so myself, but many counties still found it unusable for realtime applications.

Bottomline: progress is made slowly, often  2 steps forward and one back.  And learn from mistakes, because as my example shows, they'll stick in your memory for the rest of your life.

FSA Makes Progress

I commend the MIDAS people for displaying metrics on their posts (i.e., how many visits the page has seen and how many today).  This update from the manager is an example.

Program Time and Real Time

This excerpt from notice PL-251  spurs my memory:

In the past, changes made to data related to “actively engaged in farming”, cash rent tenant, member information, etc., simply overrode the previous information recorded. But since the determinations related to the information collected on CCC-902 are continuous, the concept of the Business File application is that the determinations are effective for a specific period of time until the plan is revised and new determinations are made by COC. At any point, users should be able to view the producer’s farm operating plans to get an historical view of the changes made to the operation.

This concept is similar to new functionality that is being developed through the MIDAS effort so it is important that State and County Office users understand the concept and how changes made in error affect the historical information available. The reason a “delete” option is not available in the system is because of the need to maintain the historical data.

The problem in the data model always is mapping reality to program requirements.  In the real world, a death, a change of ownership, etc. occurs at any time, which to me argues for specifying the date of such change. But then the question is how the event is applied to program requirements, which is another kettle of fish, and not an easy set of fish to fry.

When we automated operations on the System/36 we seemed to have two options offered by the IT types: continuous files (i.e., name and address, farm file) or year-specific files (acreage reporting, contracts). That meant the clerk/PA had to translate in her head the significance of new/changed data into the meaning for the applicable program and year.  Experience showed those didn't work well enough, so we gradually added time data to some (beginning and end years in the farm producer file) and created year-specific records within others (eligibility file).  That helped, but changing file structure was always a major commitment of programming and testing time, so it was hard to justify with our other priorities.


I wonder: is Ctrl-Z a standard in MIDAS, because while good data should be retained, there's always the need to back out bad data.

USDA Performance Measures

Forgive my asking,but aren't we now in FY13, nearing the end of the first quarter?  Having nothing better to do, I was trolling through the performance.gov site for USDA.  

When I copy the data over, I lose the formatting but these relate to MIDAS and OCE, and they seem to be a tad out of date. Tsk, tsk.

Initial Operating Capability (IOC) Deployment

IOC Release 1 acquisition and detailed planning will occur in FY11 Q3-Q4. Releases will follow SAP ASAP methodology (project prep, blueprint, realization, final prep and go live). Less

2011-07-01

2012-09-30

$119098

% of Field Offices with WAN Acceleration Monthly % 80 -- 2012-02-28 % of Field Office with centralized file services Monthly % 55 -- 2012-02-28 % of Field Office on centralized backup Monthly % 55 -- 2012-02-28 % of infrastructure components out of life cycle (goal is to refresh prior to out of life cycle so l More.. Monthly % 55 -- 2012-02-28 % End Users Satisfied with OCE-related IT Infrastructure Components Monthly % 55 -- 2012-02-28 % of Field Offices with Telephone systems upgraded Monthly % 75 -- 2012-02-28

The question is, is anybody looking at the measures?

The Progress of New Terminology and Technology

Reading Notice CP-686 on the forthcoming use of MIDAS with GIS for acreage reporting, replacing CARS. 

Two terms new to me: "subfield" and "cross-over commodity".

Remembering the fiasco of the ASCS-578 in 1985 (and 86, and 87) I wish them luck.  Actually, I hope over the years the number of problems has been reduced, but acreage reporting was probably the  area where the conflict between national standards and local conditions was most obvious. Before computers, much of the conflict was hidden from the national office; State and county offices made things work.  Introduce the computer and local variation becomes a problem.

I suspect, without any evidence whatsoever, that part of the resistance to "electronic health records" on the part of doctors and others is based on this sort of thing. 

Super User Boot Camp and the History of Training

There was a super-user boot camp for MIDAS last week.  Some 60 super-users were trained on it.  Apparently the Deputy Administrator was opening the session, because the website shows a picture of him, but the associated link points back to the Administrator's message of August.

I'm a bit curious as to the setup--whether this is train-the-trainer?  When I moved to the program side, the standard for training was: Washington program specialist trained state program specialist who trained the county CED's and PA's.  That's the way we trained for the System/36, though the "program specialists" were mostly the people hired out of the county office to work in DC (today's business process analysts, I think).  As time went on we became more sophisticated in training; we even did dry runs instead of just winging it in front of the audience.  With the advent of PC's and Word Perfect our materials could be a lot prettier, though perhaps not much improved in quality.

By the early 90's we were providing our presentations on floppy disks to the state people.  And then we started to train the trainers; rather than just relying on the state specialists, we'd pull in selected county people and mix up the areas.  The theory was in part to spread the training burden, in part to encourage cross-fertilization of ideas at the county level, rather than having 50 silos of county to state communication where the major cross-fertilization occurred at the state level.  I don't remember ever doing a detailed evaluation of our methods, to see whether we really did improve county operations through such training methods.

These days, with social media, and bring your own device, I'm sure there are new possibilities for improving training.