Testing today’s modern, complex, interconnected weapon systems often requires measuring thousands of parameters and generating huge data sets. Multi-range tests are growing in frequency, requiring data to be transferred between ranges, as are joint-Service and joint test and training exercises. Managing the data from these endeavors has become a very important part of the overall testing effort. Under the auspices of the Central Test & Evaluation Investment Program (CTEIP), a Tri-Service Data Management Study (DMS) was proposed to collect and document the data management requirements and capabilities of the major Department of Defense (DoD) Test & Evaluation (T&E) ranges, conduct a gap analysis between those requirements and capabilities, and to recommend a set of solutions, if applicable. To those ends, EMC FTRSEG was engaged to help create and administer the DMS.
The EMC FTRSEG group was tasked to develop, administer and analyze a study that covered the test range activities across the Army, Navy and Air Force. The scope of the study was very ambitious; CTEIP wished to gather data across multiple services and a very large number of attributes. This involved:
- Determining the scope of the effort
- Choosing the locations for the study
- Developing the questionnaire
- Creating the means for delivering the study
- Administering the study
- Collating the results
- Performing sophisticated statistical analyses on the data and performing a gap analysis between requirements and capabilities
- Developing a detailed roadmap for improvement
EMC FTRSEG held extensive interviews with the study sponsor to fully understand the scope and direction for the study. The EMC team then designed the automated questionnaire which had a very complex, multi-threaded format with multiple paths. This study was designed like a marketing survey, albeit a very large one. The final survey had a total of 4425 questions broken into five tracks; for comparison, the largest previous study had had a total of 368 questions.
The study was designed to be administered on-line which posed unique challenges: most of our sites did not have internet connectivity. EMC built a web front-end that connected to a SQL database back-end and ran this as a standalone application on a laptop. These laptops were then taken out to the participants at the study sites for response entry. Responses were then correlated after the fact and merged into a master database from which analyses could be run. Web access was also enabled for those participants who
could access the internet.
A master list of 18 DoD facilities was identified for the DMS site visits, six each from the Air Force, Army, and Navy. Additional DoD T&E organizations were invited via phone solicitation and emails to also participate on a voluntary basis through the web site. A total of 328 participants from 23 ranges engagedin the survey, generating almost 214,000 responses.
Sophisticated statistical analyses were then run on the responses and these were then analyzed for gaps between requirements and capabilities, now and in the future. Potential solutions to these gaps were then identified and the gaps and solutions were assessed for factors pertinent to CTEIP. A roadmap detailing how to address these gaps was created including detailed information on the proposed solutions and the project plans for how they could be implemented.
EMC FTRSEG provided the customer with an extremely robust study vehicle that yielded an immense amount of data for examination. The versatile nature of the web front-end permitted response acquisition from both remote sites with internet connectivity and from ranges isolated from the web. The statistical analyses generated detailed information about the responses and facilitated the gap analysis that followed. Sixteen pertinent gaps were identified and solutions proposed to address them. The final report and the roadmap were accepted by the sponsor and delivered to CTEIP for use in their modernization efforts. The study was a resounding success and fully satisfied the demanding requirements of the sponsor.