How can I avoid polling when fetching Outlook appointment data for add-in

I'm making an outlook add-in inside a web app we already have (React app on vite, using React 18). I looked for hooks or ways to get the data from outlook (title, attendees, times, everything for a booking really) but the handlers do nothing and the best way I've found so far is use polling but that seems like an inefficient use of ressources. My intention is to listen to start/end time changes, attendee changes, title(subject) changes so I can reflect that in my add-in, I only want the event the user is currently working on so to the best of my understanding the Graph API will not work as the event has not yet been created ( I could be mistaken here, this is what I've seen so far).

I've tried to add an async event handler on the mailbox like so:

useEffect(() => {
        const mailbox = Office.context.mailbox;
        if(mailbox.item?.itemType==='appointment'){
            mailbox.addHandlerAsync(Office.EventType.AppointmentTimeChanged, console.log('item changed'), (result) => {
                console.log(result);
            });
        }
    }, []);

event is never triggered again. My thinking was the event handler only needs to be added once but that seems incorrect. I've also tried to put the data directly in the dependency of the useEffect block (I didn't think this would work, tried it anyways) like so and it also does nothing.

    useEffect(() => {
      console.log('start time changed', Office.context?.mailbox?.item?.start);
    }, [Office.context?.mailbox?.item?.start]);

Officejs is installed on the project and I can get the add-in to show in Outlook, I just hate the idea of putting the load of polling on a user's device as some will have older/worse hardware, there has to be a better way, something like an event I can subscribe to when a change is made to avoid polling.