Currently the Web UI only shows your usage stats as daily view. But users also want to know their usage patterns over a week, month or year.
It would be useful if a week, month and year view were also included in the activity tab to see trends and usage for a longer period of time. I really can’t know if I’m becoming less or more productive over time, and what the major time wasters are in the current format.
A week summary should currently be possible to implement, but our transformation code in aw-server is sadly slow when computing summaries of a full month, let alone a whole year.
We have plans to implement this in the long run though and have this rather highly prioritized.
I have re-written aw-server in the Rust programming language (https://github.com/ActivityWatch/aw-server-rust) which will make the queries to aw-server significantly faster, but year-long requests still takes a while even on fast computers. There’s a lot of data during a whole year, so I’m not sure if we’ll be able to make it much faster than that though.
Another improvement would also be to cache queries with time spans of a month or more, I did a prototype of that a long time ago but wasn’t satisfied with the quality of the code though so it is no longer available.
A week summary could be enough, and maybe adding up several weeks summaries could do the trick for month or year summaries. I mean, building a month summary only on the basis of several week summaries. Dunno if it’d make things easier!
Great Job anyway! The best piece of code to watch my daily workload!
First of all, great project! Pretty much was I was looking for.
What is the primary purpose of this software then in your opinion?
Personally, I want to spot patterns and trends in my behaviour. For this, weekly, monthly and yearly data is super important. I’m not sure about the underlying data model. But if query duration is a problem, you can mitigate this by updating a summary table as you go. In SQL, this is typically done with trigger. Alternatively, every day, week or month the data of the previous day, week or month could get aggregated and added to the summary, instead of at each data insertion. Would that help with that?
Agreed, this is the primary case we create ActivityWatch for and one of the most important features to develop to us.
I wish it was this simple.
This is not possible for multiple reasons, primarily because the browser can not know if the user is either afk nor is currently focusing the browser window. So then you might think “Well, just save the data already aggregated” and while that might work we wouldn’t have saved the activity in a losslessly which means if we did an bad aggregation 2 weeks ago and now realized that the code for the aggregation was wrong we cannot fix the broken data. Having a single aggregation method also makes the software very unflexible.
This is our plan and while it might sound simple it is not for the following reasons:
Aggregation of more than a week is very slow on normal computers even with optimized algorithms, we would need to cache them and do aggregations of aggregations it to be scalable which is complicated
Should we do this cached aggregation on the server-side or client-side? (Personally I root for server-side)
You need to invalidate that aggregated data if the user go back in time and modify/delete an event
I will start implementing a weekly view somewhat soon as a MVP and after that start thinking about how to do this aggregated caching.
If you are eager to get long-time analysis going and don’t care about performance, it is possible to simply copy the queries in aw-webui and set the time interval to whatever you like. Or write a query yourself. @ErikBjare has a few scripts in aw-research which might be useful.
Thanks for the thorough explanation. I understand all of your concerns. I’m of a practical nature. If you have alternative plans in the future on how to make this work, I’d love to have that in a product of course. Personally, if I have the choice with potentially slightly inaccurate aggregated views and no yearly views at all, I’d pick the former. I could imagine a huge banner on the yearly plots.
So if the server rewrite doesn’t help and future plans don’t work out, I’d suggest still implementing aggregations. You might even store it aggregated only. If old events are deleted, don’t recompute or don’t store them for > 1 year, for example. I agree that it should be done server side.
Of course I’d hope to make everything work out with keeping the data.
Actually, I have some good news after the weekly prototype.
I assumed that the performance was too slow because the normal view would take very long when hacking it to show a longer period of data. But after I made a separate weekly view which did not include a timeline of events I realized that what’s actually slow is not the aggregation itself, but transmitting that whole timeline of every event (I thought it was the database and the aggregation that was slow to fetch so many events, but apparently it was mostly serializing them to JSON and sending them to the browser which was slow).
So a simple version of this feature might actually land pretty soon. Monthly summaries should be fast on pretty much all computers, yearly will still take some time but when we go over to the new aw-server implementation in rust yearly summaries should perform well.