🙏 State stuff

[svrooij]

Describe the feature

Have some sort of understanding of state in Maester would be great, a lot of people are asking about: "what has changed"? and "Can I see new failing tests?" and "Can I ignore tests"?

I want maester to have understanding of state, and introduce new states fixed, failing, new-failing

For the first version I propose to accept the previous json result as "previous state", and I also want to make a second state store using GitHub Issues. Also this should guide others in setting up their own state stuff, like having an interface.

Impact and importance

Having state is a very big ask, so I guess it is important for a lot of people. Having the report tell you which issues have been fixed should give a great feeling.

Alternatives you've considered

No response

Additional context

I'll be building this with the help of AI, and validating whats built myself

I want Maester to have a state component with the following requirements:

Save current state Compare to previous state The reports should be expended with test that suddenly fail, test that stopped failing (are fixed) and new tests specify the previous json as one way to compute the state Using github issues in a secific repository as a second way to compute state, one issue per failing test, not sure if I want issues for test that pass, maybe that should be an option Plan first and ask questions/confirmation before coding

When using github issues, set the title to the name of the test, tag the issue with the tags from that test (orca, cis, ....). With github issues set the description to the test output, and if the issue is closed by the user. You can mark this test as ignored.

[svrooij]

To give an impression of what I think would be really good is this:

1. You have the previous testresults-....json in the test-results folder.
2. Run Maester with Invoke-Maester -CompareToPrevious
3. It will run as usual, and load the last version of the test results
4. Each Test block will have an extra property called PreviousResult

This property should also be in the html report and in that same report a Changed filter should be implemented. So you can easily hide all tests that have not changed.

Report changes:

Check out the passed label 😉

[jeremypot]

Wouldn't you rather have flexibility and compare it to previous results of your choosing?

Compare today with date X for example. You could achieve this by storing all results in dated folders, and compare the json results. It's probably also much easier to AI.

Of course, you could still do -comparetoprevious, but also -comparetodate.

[svrooij]

The default would be the latest file, you can off course specify which file it should compare to. Specify a date would not work with an integration with issue trackers.

I'm just launching idea balloons 😉

[SamErde]

Love the idea, but you're right, it is a BIG ask because of the challenge of tracking history (as previously discussed). I would think this kind of feature would want to rely heavily on spec-driven development so awesome idea balloons can turn into well-integrated platform features instead of prompt-driven bolt-ons. 🙌😅

[merill]

With Maester Cloud the approach I am taking is to store the raw json for each run and Maester Cloud will have the ability for the user to pick which two dates they want to compare against. It even has a view that shows the different dates it flipped.

However for local runs I like the simplicity of @svrooij's approach of simply passing in a previous state. If the user decides what the previous date is, then we also need to include the date we are comparing against.

I do have plans for an Invoke-MaesterInteractive for Maester 3.0 which will bring in the ability for full blow local UX which should allow the user to do some of the Maester Cloud functionality on their desktop as long as they have the history of the previous runs available somewhere.

So many idea. LFG!! 😂