Category Archives: Computers

Computational Military Reasoning (Tactical Artificial Intelligence) Part 1

I coined the phrase ‘computational military reasoning’ in grad school to explain what my doctoral thesis in computer science was about. ‘Computational reasoning’ is a formal method for solving problems (technically, you don’t even need a computer). But, for our purposes it means a computer solving ‘human-level’ problems. A classic example of this would be calculating the fastest route on a map between two points. In computer science we call this a ‘least weighted path’ algorithm and I did my Q (Qualifying) Exam on this subject. I have also written extensively on the subject including these blog posts.

So, ‘computational military reasoning’ is a, “computer solving human-level military problems.” Furthermore, we can divide computational military reasoning into two subcategories: strategic and tactical (Russian military dogma also adds a third category, ‘grand strategy’); however, for now, let’s concentrate on tactical artificial intelligence; or battlefield decisions.

Tactical AI is divided into two parts: analyzing – or reading – the battlefield and acting on that information by creating a set coherent orders (commonly known as a COA or Course of Action) that exploit the weaknesses in our enemy’s position that we have found during our battlefield analysis.

 

It is said that as Napoleon traveled across Europe with his staff he would question them about the terrain that they were passing; “Where is the best defensive position? What are the best attack routes?” Where would you position artillery? What ground is favorable for cavalry attack?”

We take it for granted that such analysis of terrain and opposing forces positioned upon it is a skill that can be taught to humans. My doctoral research 1)TIGER: An Unsupervised Machine Learning Tactical Inference Generator; This thesis can be downloaded free of charge here. successfully demonstrated the hypothesis that an unsupervised machine learning program could also learn this skill and perform battlefield analysis that was statistically indistinguishable2)Using a one sided Wald test resulted in  p = 0.0001.In other words, it was extremely unlikely that TIGER was ‘guessing correctly’. from analyses performed by Subject Matter Experts (SMEs) such as instructors at West Point and active duty combat command officers.

Supervised & Unsupervised Machine Learning

Netflix recommendations are a supervised learning program. Every time you ‘like’ a movie the program ‘learns’ that you like ‘documentaries’; for example. Any program that has you ‘like’ or ‘dislike’ offerings is a supervised learning program. You are the supervisor and by clicking on ‘like’ or ‘dislike’ you are teaching the program.

TIGER is an unsupervised machine learning program. That means it has to figure everything out for itself. Rather than being taught, TIGER is ‘fed’ a series of ‘objects’ that have ‘attributes’ and it sorts them into like categories. For TIGER the objects are snapshots of battlefields.

Screen capture from TIGER. An ‘object’ has been loaded into TIGER for analysis; in this case a ‘snapshot’ of the battle of Antietam at 1630 hours on September 17, 1863. Click to enlarge.

How TIGER perceives the battlefield

When you and I look at a battlefield our brains, somehow, make sense of all the NATO 2525B icons scattered around the topographical map. I don’t know how our brains do it, but this is how TIGER does it:

Screen capture from TIGER: How TIGER converts unit positions into lines and frontages using a Minimum Spanning Tree (MST). Click to enlarge.

By combining 3D Line of Sight with Range of Influence (how far weapons can fire and how accurate they are at greater distances displayed, above, with lighter colors) with a Minimum Spanning Tree algorithm3)Kruskal’s algorithm, https://en.wikipedia.org/wiki/Kruskal%27s_algorithm the above image is how TIGER ‘sees’ the battlefield of Antietam. This is an important first step for evaluating object attributes.

How to determine the attribute of anchored or unanchored flanks

Battlefields are ‘objects’ that are made up of ‘attributes’. One of these attributes is the concept of anchored and unanchored flanks. While anyone who plays wargames probably has a good idea what is meant by a ‘flank’, following formal scientific methods I had to first prove that there was agreement among Subject Matter Experts (SMEs) on the subject. This is from one of the double-blind surveys given to SMEs:

Screen shot from online double-blind survey of Subject Matter Experts on identifying the presence of Anchored and Unanchored Flanks. Click to enlarge.

And their responses to the situation at Antietam:

Subject Matter Experts response to the question of the presence of Anchored or Unanchored flanks at Antietam. Click to enlarge.

And another double-blind survey question asked of the SMEs about anchored or unanchored flanks at Chancellorsville:

Response to double-blind survey question asked of SMEs about anchored and unanchored flanks at Chancellorsville. Click to enlarge.

So, we have now proven that there is agreement among Subject Matter Experts about the concept of ‘anchored’ and ‘unanchored’ flanks and, furthermore, some battlefields exhibit this attribute and others don’t.

Following is a series of slides from a debriefing presentation that I gave to DARPA (Defense Advanced Research Projects Agency) as part of my DARPA funded research grant (W911NF-11-200024) describing the algorithm that MATE (the successor to TIGER) uses to calculate if a flank is anchored or unanchored and how to tactically exploit this situation with a flanking maneuver. This briefing is not classified. Click to enlarge slides.

How to generate a Course of Action for a flank attack

Once TIGER / MATE has detected an ‘open’ or unanchored flank it will then plot a Course of Action (COA) to maneuver its forces to perform either a Turning Maneuver or an Envelopment Maneuver. Returning to the previous DARPA debriefing presentation (Click to enlarge):

MATE analysis of the battle of Marjah (Operation Moshtarak February 13, 2010)

The following two screen captures are part of MATE’s analysis of the battle of Marjah suggesting an alternative COA  (envelopment maneuver) to the direct frontal assault that the U. S. Marine force actually performed at Marjah. Click to enlarge:

Conclusions & Comments about Computational Military Reasoning (Tactical Artificial Intelligence) & Battlefield Analysis (Part 1)

Usually, at this point when I give this lecture, I look out to my audience and ask for questions. I really don’t want to lose anybody and we’ve got a lot more Tactical AI to talk about. So far, I’ve only covered how my programs (TIGER / MATE) analyze a battlefield in one particular way (does my enemy – OPFOR in military terms – have an exposed flank that I can pounce on?) and there is a lot more battlefield analysis to be performed.

It’s easy, as a computer scientist, to use computer science terminology and shorthand for explaining algorithms. But, I worry that the non computer scientists in the audience will not quite get what I’m saying.

Do you have any questions about this? If so, I would really like to hear from you. I’ve been working on this research for my entire professional career (see A Wargame 55 Years in the Making) and, frankly, I really like talking about it. As a TA said to me many years ago when I was an undergrad, “There are no stupid questions in computer science.” So, please feel free to write to me either using our built in form or by emailing me at Ezra [at] RiverviewAI.com

References   [ + ]

1. TIGER: An Unsupervised Machine Learning Tactical Inference Generator; This thesis can be downloaded free of charge here.
2. Using a one sided Wald test resulted in  p = 0.0001.In other words, it was extremely unlikely that TIGER was ‘guessing correctly’.
3. Kruskal’s algorithm, https://en.wikipedia.org/wiki/Kruskal%27s_algorithm

A Wargame 55 Years in the Making (Part 5)

In June, 2009 I successfully defended my thesis and was awarded a doctorate of computer science by the University of Iowa. What followed were some of the most productive years of my professional career. I designed, programmed, project managed and was principal investigator on:

MARS: Military Advanced Real-time Simulator (2009)

OneSAF is the “Semi Automated Forces” wargame / simulator used for training by the US Army. It relies on ‘pucksters’ (see pucksters in this blog) who are usually retired military officers who make all the moves for OPFOR (Opposition Forces), MARS provided an intuitive Graphical User Interface (GUI) for the modification and running of OneSAF scenarios.

Screen capture of the MARS project for the US Army. MARS was a front end to facilitate creating and managing scenarios run on the Army's OneSAF military simulator. Click to enlarge.

Screen capture of the MARS project for the US Army. MARS was a front end to facilitate creating and managing scenarios run on the Army’s OneSAF military simulator. The ‘Magic Bomb’ option is the puckster’s term for ‘magically’ removing a unit from the simulation. Click to enlarge.

CAPTURE: Cognitive and Physiological Testing Urban Research Environment (2010)

While on the surface CAPTURE appears to be a standard ‘shooting gallery’ program it was actually designed to test and store data about how returning veterans saw targets, ‘spiraled in’ on targets and reacted. There were two parts to CAPTURE: the first allowed the tester to set up any particular scenario they wanted (top image, below) and the second part (bottom image, below) was run using a projector, a large screen, an M16 air soft gun with Wii remote and laser mounted to the barrel and an IR camera. CAPTURE was done for the Office of Naval Research (Marines).

Two screens showing the CAPTURE program. The top screen shows the interface for creating target scenarios. The bottom screen is one of the the shooting ranges generated by CAPTURE. Click to enlarge.

Two screens showing the CAPTURE program. The top screen shows the interface for creating target scenarios. The bottom screen is one of the the shooting ranges generated by CAPTURE. Click to enlarge.

NexGEN Behavior Composer (2011)

NexGEN Behavior Composer was another front-end project for OneSAF. Enemy units in OneSAF use scripted AI behavior written in XAML. These AI scripts often contained errors. NexGEN allowed the puckster to select a behavior from a hierarchical tree structure (top image, below) and click and drag it to a composing canvass where a series of behaviors could be joined together (bottom image, below). The artwork for the behaviors was done by my old friend, Ed Isenberg, who has worked with me on games since the ’80s.

Screen shot of NexGEN Behavior Composer which facilitated creating OneSAF behaviors by clicking and dragging behavior icons. Click to enlarge.

Screen shot of NexGEN Behavior Composer which facilitated creating OneSAF behaviors by clicking and dragging behavior icons. This is the hierarchical tree structure of behavior primitives. Click to enlarge.

And example of a OneSAF behavior composed of NexGEN behavior icons. Click to enlarge.

A series of behaviors have been placed together to create a complex behavior (a unit fires, conducts reconnaissance, waits for one minute, moves and then occupies a position). Click to enlarge.

MATE: Machine Analysis of Tactical Environments (2012)

Funded by a DARPA (Defense Advanced Research Project Agency) research grant (W911NF-11-200024) MATE added a new level of battlefield analysis to the TIGER project. Building on the previous nine years of research MATE had the capability of generating a series of ‘predicate statements’ that described the battlefield and then using them to construct a hypothetical syllogism that resulted in a precise Course of Action (COA) for BLUEFOR (US forces). MATE then output this COA as an HTML file and automatically launched a browser to view the COA. MATE was designed to be available to commanders in the field via a small handheld device like a tablet. It was able to perform battlefield analysis in less than 10 seconds.

Consider this real-world situation from the Battle of Marjah:

Given the same data that the commander had in the above video MATE returned this COA (complete with unit paths and ETAs):

MATE's analysis and COA for the Battle of Marjah: a right-flank envelopment maneuver with two infantry platoons while a fixing force of the mortar platoon and a third infantry platoon kept the enemy's attention. Click to enlarge.

MATE’s analysis and COA for the Battle of Marjah: a right-flank envelopment maneuver with two infantry platoons while a fixing force of the mortar platoon and a third infantry platoon kept the enemy’s attention. Click to enlarge.

To see the entire MATE analysis and COA results for the Battle of Marjah click here. (this will load a PDF of MATE’s HTML output on a new tab).


Unfortunately, about the time that I demonstrated MATE to DARPA a series of unfortunate events occurred that were to change my life. The United States Congress passed the Sequestration Transparency Act of 2012. This resulted in a 10% across the board cut in all federal spending. DARPA seemed especially hard hit and they stopped all funding for 4CI (Command, Control, Communications, Computers and Intelligence) research. Only a few years after receiving my doctorate, specifically so I could be the Principal Investigator on government funded 4CI research, I was out of a job.

Without any research funding, and not wanting to relocate I returned to the University of Iowa as a Visiting Assistant Professor teaching Computer Game Design and CS1.  I love teaching. And I am extraordinarily proud of receiving the highest student evaluations in the department of Computer Science but I didn’t have as much strength as I used to have. I found myself out of breath and exhausted after a lecture. And then my kidneys began to inexplicably fail.

In 2013, because of the efforts of superb doctors Kelly Skelly and Joel Gordon at the University of Iowa Hospital, I was diagnosed with a very rare and usually fatal blood disease, AL amyloidosis.  In 2014, thanks to the Affordable Care Act, I was hospitalized for 32 days, my immune system  was purposely destroyed and I received an autologous bone marrow / stem cell transplant. This was followed up by a year of chemotherapy. Being severely immunocompromised I have contracted pneumonia six times in the last two years. Now, against the odds (and I’m a guy that deals with probabilities a lot so I’m being literal) I’ve completely recovered. My kidneys and lungs are permanently damaged but I’m not going to die from this disease. But, it also means I can’t teach anymore, either.

Luckily, I can still sit at a desk and write computer code. General Staff is my return to writing a commercial computer wargame and it will be the first commercial implementation of my tactical AI algorithms that I have been developing since 2003.

I need to produce a game that you grognards want. And, that means next week I will be posting a new gameplay survey to pin down exactly what features you want to see in the new game. As always, please feel free to contact me directly (click here) if you have any questions or comments.

A Wargame 55 Years in the Making (Part 3)

The goal of my doctoral research was to create a suite of algorithms that were capable of making ‘human-level’ tactical and strategic decisions. The first step is designing a number of ‘building block’ algorithms, like the least weighted path algorithm that calculates the fastest route between two points on a battlefield while avoiding enemy fire that we saw in last week’s post. Another important building block is Kruskal’s Minimum Spanning Tree algorithm which allows the computer to ‘see’ lines of units.

I use terms like ‘see’ and ‘think’ to describe actions by a computer program. I am not suggesting that current definitions of these terms would accurately apply to computer software. However, it is simply easier to write that a computer ‘sees’ a line of units or ‘thinks’ that this battlefield situation ‘looks’ similar to previously observed battlefields. What is actually happening is that units are represented as nodes (or vertices) in a a graph and some basic geometry is being applied to the problem. Next week we will use probabilities. But, at the end of the day, it’s just math and computers, of course, don’t actually ‘see’ anything.

Examples of how Kruskal's Minimum Spanning Tree algorithm can be used to separate groups of units into cohesive lines. These figures are taken from, "Implementing the Five Canonical Offensive Maneuvers in a CGF Environment." by Sidran, D. E. & Segre, A. M.

Examples of how Kruskal’s Minimum Spanning Tree algorithm can be used to separate groups of units into cohesive lines. These figures are taken from, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. & Segre, A. M.

When you and I look at a map of a battle we immediately see the opposing lines. We see units supporting each other, interior lines of communication, and lines of advance and retreat. The image, below, shows how the program (in this case, TIGER, the Tactical Inference Generator which was written to demonstrate my doctoral research) ‘sees’ the forces at the battle of Antietam. The thick black line is the ‘MST Spine’. You and I automatically perceive this as the ‘front line’ of the Confederate forces, but this is a visual representation of how TIGER calculates the Confederate front line. Also important is that TIGER perceives REDFOR’s flanks as being anchored (that is to say, BLUE does not have a path to the flanking objective, the tip of the green vector, that does not go through RED Range of Influence, ROI, or Zone of Control).

Figure 1. TIGER screen shot of ‘flanking attribute’ calculations for the battle of Antietam (September 17, 1862, 0600 hours). Note the thick black line that repres ents the MST spine of REDFO R Group 0, the extended vectors th at calculate the Flanking Goal Objective Point and BLUEFOR and REDFOR ROI (red and blue shading). REDFOR (Confederate) has anchored flanks.

TIGER screen shot of ‘flanking attribute’ calculations for the battle of Antietam (September 17, 1862, 0600 hours). Note the thick black line that represents the MST spine of REDFOR Group 0, the extended vector that calculates the Flanking Goal Objective Point and BLUEFOR and REDFOR ROI (red and blue shading). REDFOR (Confederate) has anchored flanks. From, “Algorithms for Generating Attribute Values for the Classification of Tactical Situations,” by Sidran, D. E. & Segre, A. M.

Now that TIGER can see the opposing lines and recognize their flanks we can calculate the routes for implementing the Course of Action (COA) for various offensive maneuvers. U. S. Army Field Manual 3-21 indicates that there are five, and only five, offensive maneuvers. The first is the Penetration Maneuver (note: the algorithms for these and the other maneuvers appear in, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M.) and can be downloaded from ResearchGate and Academia.edu.

The Penetration Maneuver is described in U.S. Army Field Manual 3-21 and as implemented by TIGER. Note how TIGER calculates the weakest point of REDFOR's line. From, "Implementing the Five Canonical Offensive Maneuvers in a CGF Environment." by Sidran, D. E. and

The Penetration Maneuver is described in U.S. Army Field Manual 3-21 and as implemented by TIGER. Note how TIGER calculates the weakest point of REDFOR’s line. From, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M. Click to enlarge.

The next maneuver is the Infiltration Maneuver. Note that to implement the Infiltration Maneuver, BLUEFOR must be able to infiltrate REDFOR’s lines without entering into RED’s ROI:

The Infiltration Maneuver.

The Infiltration Maneuver as described in U.S. Army Field Manual 3-21 and as implemented by TIGER. Note how TIGER reaches the objectives without entering into REDFOR ROI. From, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M. Click to enlarge.

The next maneuver is the Turning Maneuver. Note: in order to ‘turn an enemy’s flanks’ one first must be able to recognize where the flanks of a line are. This is why the earlier building block of the MST Spine is crucial.

The Turning Maneuver as illustrated in U. S. Army Field Manual 3-21 and in TIGER.

The Turning Maneuver as illustrated in U. S. Army Field Manual 3-21 and in TIGER. From, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M. Click to enlarge.

Certainly the most complex offensive maneuver is the Envelopment Maneuver which requires two distinct movements and calculations for the attacking forces: first the attacker must decide which flank (left or right) to go around and then the attacker must designate a portion of his troops as a ‘fixing force’. Think of an envelopment maneuver as similar to the scene in Animal House when Eric “Otter” Stratton (played by Tim Matheson) says to Greg Marmalard (played by James Daughton), “Greg, look at my thumb.” Greg looks at Otter’s left thumb while Otter cold-cocks Marmalard with a roundhouse right. “Gee, you’re dumb,” marvels Otter. In an envelopment maneuver the fixing force is Otter’s left thumb. Its purpose is to hold the attention of the victim while the flanking force (the roundhouse right) sweeps in from ‘out of nowhere’. In the next post I will show a real-world example of an Envelopment Maneuver created by my MATE (Machine Analysis of Tactical Environments) program for DARPA.

The Envelopment Maneuver as shown in U. S. Army Field Manual 3-21 and as implemented in TIGER.

The Envelopment Maneuver as shown in U. S. Army Field Manual 3-21 and as implemented in TIGER. From, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M. Click to enlarge.

Lastly, and obviously the maneuver of last resort, is the Frontal Assault:

The Frontal Assault Maneuver from

The Frontal Assault Maneuver from, “Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.” by Sidran, D. E. and Segre, A. M. Click to enlarge.

All that I’ve done in this post is show some of the things that the TIGER program does. What I haven’t done is show how the algorithms work and that’s because they are described in the papers, below. Obviously, this is a subject that I find pretty interesting, so feel free to ask me questions (you can use the Contact Us page).

It is my intention to incorporate these algorithms into the General Staff wargame. However, I’ve been told by a couple of game publishers that users don’t want to play against a human-level AI. What do you think? If you’ve read this far I would really appreciate it if you would answer the survey below.


Papers that were cited in this post with download links:

“An Analysis of Dimdal’s (ex-Jonsson’s) ‘An Optimal Pathfinder for Vehicles in Real-World Terrain Maps'”

In PDF Format

“Algorithms for Generating Attribute Values for the Classification of Tactical Situations.”

In PDF Format

“Implementing the Five Canonical Offensive Maneuvers in a CGF Environment.”

In PDF Format


 

Rules for Artillery

Screen shot of the page describing the rules for artillery in General Staff.

Screen shot of the page describing the rules for artillery in General Staff.

The special, “proof of concept” demo that we’ve been working for a ‘very well known computer wargame publisher’ is coming along very well and we wanted to share this screen capture of the Special Artillery Rules page. This also will give you a glimpse into how artillery will be used in the game.

As always, comments, critiques (and catching typos!) are always welcome!

New, Faster Pathfinding AI

A screen shot showing traditional A* (pronounced A Star) pathfinding. The green areas are 'nodes' that the algorithm explored on its way to finding the optimal path (in Brown).

A screen shot showing traditional A* (pronounced A Star) pathfinding. The green areas are ‘nodes’ that the algorithm explored on its way to finding the optimal path (in Brown). Click on picture to enlarge to full size.

Artificial Intelligence (AI) plays an important role in wargame development; it’s what separates a good game from a great game. One of the most important algorithms employed in wargame AI is the A* (pronounced ‘A star’) pathfinding algorithm that was created in 1968 by Peter Hart, Nils Nilsson and Bertram Raphael. The paper describing it, A Formal Basis for Heuristic Determination of Minimum Cost Paths can be downloaded here. I did my doctoral Qualifying Exam on optimized pathfinding. My paper, “An Analysis of Dimdal’s (ex-Jonsson’s) ‘An Optimal Pathfinder for Vehicles in Real-World Terrain Maps'” can be downloaded here.

How long will it take for your orders to arrive?

How long will it take for your orders to arrive at this unit? How long will it take for the unit to send a courier back to headquarters with its current location?

Pathfinding is important in wargames because it’s how units, under computer control, move around on the map. Also, and we’re announcing this for the first time here, when you give orders in General Staff a courier has to ride from your headquarters unit to the unit that is to receive your orders. Also, units on the battlefield that are not directly visible to the Headquarters unit (this is done with a 3D Bressenham line algorithm; more about this later) slowly begin to fade from view on the map. However, every hour a courier is dispatched from every unit to headquarters with an update on their position. As we can see from the information box, above, the courier will take 41 minutes to deliver the new position information to headquarters.

The top screen capture shows an implementation of the classic A* algorithm for calculating the optimal path from Blue’s headquarters unit to a far-flung cavalry unit. Note, this is an especially difficult path to calculate because the unit is across a river and there are only three bridges across. The A* algorithm performs perfectly but it is just too slow to be used with a real-time tactical wargame like General Staff. After some thought I wrote a major optimization of A* which we present here for the first time.

An example of the new EZRoadStar pathfinding algorithm created for General Staff. Compare it to the top screen capture which uses the classic A* algorithm. Click to enlarge.

An example of the new EZRoadStar pathfinding algorithm created for General Staff. Compare it to the top screen capture which uses the classic A* algorithm. Click to enlarge.

Above is a screen shot of the results of the new EZRoadStar algorithm. It is almost identical to the original A* algorithm but runs thousands of times faster (my fellow computer scientists would probably prefer if I did some tests, wrote a paper and published the exact figures and I promise I’ll get around to that, some day).

In the screen shot, above, you can see the path of the courier (in green) from the Blue HQ unit to wayward cavalry unit. The new pathfinding algorithm, EZRoadStar, first looks for roads and then calculates how to get on and off the roads. This is much faster than the A* algorithm.