Home
People
Research
Publications
Events
Links
 
 

Trajectory-Based Analysis of Team Sports

Activity recognition

Once the game is segmented into individual game phases, we have to determine the type of each individual segment of the game. We have developed a template-based activity recognition method which is used to:

  1. to recognize the type of an unknown activity by using an activity template or a previously recognized activity,
  2. to cluster together similar activities that are observed during the game,
  3. to solves an important combinatorial problem of determining the relations between the roles of the players between the two observed activities or between activity and template.

Activity template

Basketball coaches often use activity templates when they want to develop new game strategies and to pass their knowledge to players. These templates encode the spatio-temporal properties of different basic game elements which are the building blocs of every basketball activity. These elements are: screens, starting formations, ball passes, player motion around the court, dribbling and other (Figure 1).

 
Figure 1: An example of a basketball activity template called "Flex".
 
By using the information from the templates, we are able to generate the simulation of the activity which is then uses in the process of activity recognition.
 
 
Figure 2: Simulation of "Flex" activity.
 

Problems concerning group activity recognitions

The problems concerning group activity recognition are threefold:

  • a very high spatial and temporal variability in player trajectories (Figure 3).
  • combinatorial problems of determining the player roles in two consecutive activities or an activity and template.
  • lack of large number of consistent training examples from which a statistical model of an activity could be learned.
 

An example of a "Flex" basketball offense:

Player trajectories of several "Flex" offenses:

Temporarily aligned trajectories using Dynamic Time Warping:

 
Figure 3: An example of analyzed game data.
 
 
Recognition of basketball activities
 

Semantic description of a trajectory segment is used to compare the trajectory segment with the activity template. The matching process is done in two steps:

  • First we need to determine the relations between players on the court and the player indexes in the activity template. We call this process player casting.

  • After the roles of individual players are obtained, we can determine which of the template-generated semantic descriptions
    is the most similar to the trajectory-generated description.
1. Player casting

To solve the problem of casting, we have to test 120 (5!) possible casts, and select the permutation, which yields the smallest distance to the semantic description of the compared activity template. For each possible permutation (cast), we first generate five distinctive player agendas, one for each of the players. One set of agendas is extracted from the trajectory-generated semantic descriptions, and the other set from the template-generated semantic descriptions. Each player agenda contains only those activities, in which the player in question has participated (Table 2).

 

(1) FORM_flex1
(2) P3_M_LEFT_CORNER_LEFT_POST
(3) SCR_2_3_LEFT_POST
(4) P1_M_RIGHT_POST_RIGHT_CORNER
(5) P3_M_LEFT_POST_KEY
(6) P4_M_LEFT_WING_LEFT_POST
(7) SCR_2_4_LEFT_POST
(8) P2_M_LEFT_POST_LEFT_WING
(9) P3_M_KEY_RIGHT_POST
(10) FORM_flex1inv
(11) P1_M_RIGHT_CORNER_RIGHT_POST
(12) P5_M_RIGHT_WING_PIVOT

(13) SCR_1_3_RIGHT_POST
(14) P5_M_PIVOT_RIGHT_POST
(15) P1_M_RIGHT_POST_KEY
(16) SCR_3_5_RIGHT_POST
(17) P3_M_RIGHT_POST_RIGHT_WING
(18) P1_M_KEY_LEFT_POST
(19) SCR_1_4_LEFT_POST
(20) P4_M_LEFT_POST_KEY
(21) P4_M_KEY_RIGHT_POST
(22) P2_M_LEFT_WING_PIVOT
(23) SCR_4_5_RIGHT_POST

Table 1: Semantic description obtained from the "Flex" offense shown in Figure 3.
Numbers in the parenthesis state the order of elements.

 

Agenda 1

FORM_flex1
M_RIGHT_POST_RIGHT_CORNER
FORM_flex1inv
M_RIGHT_CORNER_RIGHT_POST
SCR_RIGHT_POST
M_RIGHT_POST_KEY
M_KEY_LEFT_POST
SCR_LEFT_POST

Agenda 2

FORM_flex1
SCR_LEFT_POST
M_LEFT_POST_LEFT_WING
FORM_flex1inv
M_LEFT_WING_PIVOT
Agenda 3

FORM_flex1
M_LEFT_CORNER_LEFT_POST
SCR_LEFT_POST
M_LEFT_POST_KEY
M_KEY_RIGHT_POST
FORM_flex1inv
SCR_RIGHT_POST
M_RIGHT_POST_RIGHT_WING
Agenda 4

FORM_flex1
M_LEFT_WING_LEFT_POST
SCR_LEFT_POST
FORM_flex1inv
SCR_LEFT_POST
M_LEFT_POST_KEY
M_KEY_RIGHT_POST
SCR_RIGHT_POST
Agenda 5

FORM_flex1
FORM_flex1inv
M_RIGHT_WING_PIVOT
M_PIVOT_RIGHT_POST
SCR_RIGHT_POST
 

Table 2: Player agendas built from the semantic description in Table1.


 

The 120 possible player casts, one for each possible permutation of five players, are then tested by cross-comparing five template-generated player agendas to five trajectory-generated ones.

In our work, we use the modified Levenshtein distance as a similarity measure between the two sequences of symbols. The distance measure assigns a penalty of 2 for each symbol insertion or deletion and the final distance is normalized with the sum of lengths of both symbol sequences. This way, the final output of cost function is normalized to the interval [0..1], yielding zero for a total match and one for total mismatch.

The overall cost of particular player cast (permutation) is calculated as the sum of individual cost functions for all five players. This process is repeated for each of the 120 possible permutations - casts, and the permutation, which yields the smallest cost function, is accepted as the best explanation of player roles in the observed trajectory segment.

 
2. Assigning the label to the analyzed trajectory segment.

After the player casting is completed, the symbols in the trajectory-generated sequence are updated to reflect template player assignment, and the overall cost function is calculated by comparing the trajectory-generated sequence to the template-generated one, using the modified Levenshtein distance. This process, complete with the role casting, has to be repeated for each activity template from the database.

After the results of comparison are obtained, the observed trajectory segment is assigned the label of the template that generated the smallest distance. However, this is done only if the cost is smaller then the certain threshold, which was, in our case, set to 0.6. We assume that there is no match for the observed activity in the activity database if the distance is above this value. As an illustration, Figure 4 shows the results of matching the trajectory segment, manually labeled "Flex offense" to 17 different templates, including the "Flex offense" template.

 
 
Figure 4: Recognition results when comparing the "Flex" activity to 17 different activity templates.
Second bar shows the result of the "Flex" template.
 
 
 

Recognition results

Results were obtained using Matlab 7 on Pentium 4 (2,8 GHz, 2GB of RAM).

 

Time needed to recognize an activity:   2.67 s
Average time needed to build a semantic description of a template (each description is built only once):   11.3 s
Recognition performance when no additional symbols were added into description:   100 %
Recognition results when different amounts of additional samples (noise) were added:
Amount of added symbols [%]
0
50
100
200
300
400
500
Recognition rate [%]:
100
99.0
98.9
97.5
96.2
94.8
91.8

 

previous page | next page

 

 

 
Selected publications:  
   
(1) M. Perše, M. Kristan, J. Perš, S. Kovačič."Automatic Evaluation of Organized Basketball Activity". In: Michael Grabner, Helmut Grabner (eds.), Computer Vision Winter Workshop 2007, St. Lambrecht, Austria, pp.11-18. February 2007.

 
(2) M. Kristan, J. Perš, M Perše, S. Kovačič."Towards fast and efficient methods for tracking players in sports". In: Proceedings of the ECCV Workshop on Computer Vision Based Analysis in Sport Environments, pp.14-25. May 2006.

 
(3) M. Perse, M. Kristan, J. Pers, S. Kovacic."A Template-Based Multi-Player Action Recognition of the Basketball Game". In: Janez Pers, Derek R. Magee (eds.), Proceedings of the ECCV Workshop on Computer Vision Based Analysis in Sport Environments, Graz, Austria, pp.71-82. May 2006.