Design Proposal for Compressing Events

1. At the point where the kubelet code “records/writes” the event (StartRecording in pkg/client/record/event.go):
   • Introduce a hash table to track previously fired events (where key is “event, minus timestamps and count” and value is ”number of times previously seen”).
   • For each new event:
     • If the event matches with a previously fired event (from hash table), increment the count in the hash table, and call new PUT (update) event API (instead of the POST/create event API) to update existing event entry with new last seen time stamp and count.
     • If the event does not match a previously fired event (from hash table), create an entry in the hash table for the event with an initial value of 1, and continue with the existing behavior (call POST/create event API) to create a new entry for the event.
2. In pkg/api/types.go modify the Event struct to:
   • Change Timestamp to InitialTimestamp
   • Add a LastSeenTimeStamp field (by default it will have the same value as InitialTimeStamp)
   • Add a Count field (by default it will be 1)
3. Update kubectl, to correctly print out the new/modified Event struct fields.
4. Extend the Kubernetes API to add “Update Event" support, so that an exiting instance of an EVENT can be replaced with a new instance, instead of always creating a new instance.

"Got yas" to watch out for:

• Hash table clean up.
  • If kubelet runs for a long period of time and fires a ton of unique events, the hash table could grow very large in memory.
  • Future consideration: remove entries from the hash table that are older then some specified time.
• Hash table not preserved across Kubelet restarts:
  • If Kubelet restarts, the hash table is cleared, thus, compression will not occur for events across Kubelet restarts. This shouldn't be a problem.

Does sort order for events change to "last seen"?

Good question. I was planning on leaving it as "Initial Time", but now that I think about it, it would make more sense to sort on "Last Seen Time".

We should convert this doc into something we can actually check in to the codebase under docs/design - doesn't have to be long, but should explain and be actually current.

Great point, will do.