PROJECTS WE'RE WORKING ON

NASA

NASA manages the NASA Technical Reports Server (NTRS) Database as part of the Scientific and Technical Information Program. There are hundreds of thousands of articles in the database and around two hundred new documents are published and stored every month. With a huge database that is constantly growing in volume, it can be a major challenge for engineers to sift through hundreds of articles to find the ones with the information they need to make critical mission design plans and schedules.

The Eureka Project was designed to tackle this problem by pairing intelligent minds with the power of artificial intelligence. 

The goal of the Eureka Project was to enhance the System of System (SoS) analysis toolset by integrating Artificial Intelligence (AI) and formal Model-based Systems Engineering (MBSE) in four key ways:

1. Develop and validate AI agents for the dynamic and ongoing collection of pertinent information related to the systems in each architecture, to complement and enhance information from Subject Matter Experts
2. Develop methods to rapidly ingest and analyze architectures from multiple sources including formal MBSE-based architectures and systems developed in different MBSE tools
3. Test & tune the enhanced toolset using an AI generated/curated Space Architecture Database.
4. Begin the development of a digital thread that can link technology status and architecture analysis to requirements and systems development

Each research team was working on a different mission plan, so each team broke out the components of their mission into systems and subsystems.  The teams wrote down specific descriptions for each subsystem and listed the associated Technology Area Breakdown Structure (TABS) designations. This is where the engineers would normally read through article after article trying to find the information they need to design the components and the schedule based on the Technology Readiness Level (TRL).  Instead the teams worked closely with Bintel to find the information they needed.

The Bintel Team went through the subsystems with each team and developed an agent for each subsystem using the text provided, relevant text from the TABS descriptions, and verbal guidance.  An important component of the agents is keywords, so the Bintel Team worked closely with the researchers to develop a list of keywords for each agent. In order to avoid false positives the keywords and phrases must be precise and only used for the topic the agent is being trained for.  This can be easier to do in the scientific community because terminology and word definitions tend to be more consistent than in other types of text.

The Data Analysts trained the agents against thousands of sample summary texts from other articles and returned the top twenty articles for each agent.  The research team then went through the results marking which articles were relevant and which were not. After three rounds of agent training, the agents were consistently returning relevant results.

The content of the results is important because engineers make important design decisions based off of the information they gather.  Their research needs to be comprehensive and their findings need to be correct, otherwise they will be forced to make design changes or schedule changes further along in the project, when it becomes much more expensive.  The agents found relevant articles and reports under all different TABS designations, which saved the engineers time and money while being more comprehensive than traditional research methods. The research teams were also pleasantly surprised that the agents found documents relevant to a subsystem that they intended on using, but also returned documents that provided a different option to solve the same problem.  

The Topic Agents are able to find these critical pieces of information because they use training text and keywords to find the most relevant documents.  So although the subsystem only describes a specific component or design type, the text contains information about how that component works and articles that have new methods are picked up because they are describing the same function.  The agents will help provide insight that cannot otherwise be achieved.

After rounds of training and review of results, Bintel’s Development Team and Data Scientists loaded all of the data up into the cloud and created interactive dashboards for the research teams.  This way the users could easily see all their data in a digestible format with graphics.

Key Dashboard Features Include:

• Date Histogram: can be used to view how many articles are published over a given time period, making it easier to see if a concept is well studied or new
• Organizations & Authors: shows where and who is publishing articles on a specific topic so the user can reach out to the organization or author for an in-person meeting
• Abstract Table: shows the user the abstract text for articles that are relevant to the applied filters
• Exportable Table: can be exported to a csv file so the user can send a list of articles with links to the original document in the NTRS database

The Bintel Team provided the research teams with agents for their subsystems, agents for each of the TABS designations, and an interactive dashboard.  This helped them access the wealth of knowledge that was hidden in their giant store of documents. It was a fun project for all parties involved and Bintel looks forward to continuing on with this project in the future.

Future plans include adding more agents and subsystems, and creating a web application so new teams can build their own projects with their own systems and subsystems and run the agents against a constantly updating NTRS database and additional external sources.

Learn more about this project and the challenges here.

INDUSTRY WE WORKED WITH

Aerospace

TOOLS USED FOR THIS PROJECT

HAVE A SIMILAR PROJECT AND
WANT TO WORK WITH US?