Add section on procedure

[libreriscv.git] / HDL_workflow / libresoc_bug_process.mdwn

# LibreSOC Bug Process

* [Bug #1126](https://bugs.libre-soc.org/show_bug.cgi?id=1126)

* HDL workflow guide page: [[HDL_workflow]]

This page describes in detail how to raise new tasks (bugs) and how to approach
development within the project in order to get appropriate amount of funding
for the tasks completed.

# Why raise issues

* [Bug #1126](https://bugs.libre-soc.org/show_bug.cgi?id=1126)

If you have discovered a problem in Libre-SOC (software, hardware, etc.),
please raise a bug report!
Bug reports allow tracking of issues, both to make the developers lives easier,
as well as for tracking completed grant-funded work.

It is **extremely** important to link the new bug to previous ones. As Luke
mentioned on [this bug](https://bugs.libre-soc.org/show_bug.cgi?id=1139#c3),
"it is a mandatory project requirement that the graph from any bug
contain all other bugs (one "Group")".

The primary reason for this is to ensure bugs don't get buried and lost,
and will aid those tackling similar problems at a later time.

Also, for project management and financing purposes, it helps developers
to keep an up-to-date list of their tasks.

##How to raise issues

1. Create a bug report.
2. Add in any links from the mailing list or IRC logs to the bug report for
back tracking (this is mandatory). Also fill in the URL field if there is a
relevant wiki page.
3. CC in relevant team members
4. Make absolutely sure to fill in "blocks", "depends on" or "see also" so
that the bug is not isolated (otherwise bugs are too hard to find if isolated
from everything else)
5. Ping on IRC to say a bug has been created
6. Unless you know exactly which milestone to use, leave blank initially. This
also applies when editing milestone, budget parent/child, toml fields. See
section [[HDL_workflow#Task management guidelines]] further down.
7. After setting the milestone, it is **absolutely required** to run
[budget-sync](https://git.libre-soc.org/?p=utils.git;a=blob;f=README.txt;hb=HEAD),
as it will point out any discrepancies. The budget allocations will be used for
accounting purposes and **MUST** be correct. *Note you can only get paid for
stuff done **after the nlnet grant is approved** (before the MOU is signed)*

If a developer ever needs to check which bugs are assigned to them, go to the
Libre-SOC bug tracker
[advanced search page](https://bugs.libre-soc.org/query.cgi?format=advanced),
and in the "Search by People" section, check "Bug Assignee" and "contains"

# Adding sub-tasks to tasks under existing milestone

* notify Michiel and the relevant NNNN-NN@nlnet.nl team of
  advance notice of intent to add new sub-tasks, cc'ing bob
  goudriaan 
  - confirm with them that this is NOT a change in the AGREED
    MILESTONE BUDGETs, because it is just sub-task allocation.
  - confirm that they are happy to add the sub-tasks to the MoU
    (this needs approval of bob goudriaan)
* *re-generate* the JSON file
* make a DIFF against the *PREVIOUS* JSON file
* create a MANUAL report/summary of "changes" that
  NLnet may easily action
  - "add the following task X to parent Y of amount W",
  - and if any: "change parent Z available amount to V as a WRAPUP")
  (this latter is because occasionally there are subtasks **not**
   totalling the full parent amount, usually because a summary
   report is needed. Michiel and I privately agreed to call
   this "wrapup")
* obtain a confirmation that this has been actioned
* **double-check** that the RFP database correctly matches the new
  bugzilla status.

PLEASE NOTE: YOU CANNOT ACTION THE ABOVE UNDER THE FOLLOWING CIRCUMSTANCES

1. to make a change to the actual budgetary amounts of the
   Grant Milestones, without written authorization from Bob
   and Michiel. a DIFFERENT PROCEDURE is needed.
   this is because NLnet had to go through a 3rd party Verification
   Process with the European Union: changing the amounts without
   consent is therefore tantamount to fraud.

2. if there has been an RFP already submitted under a given Milestone,
   it becomes NO LONGER POSSIBLE to change the JSON file in NLnet's
   system because it is too complex.

   there is one Grant in this complex situation: bug #690, the crypto
   grant.  it is made much more complex because it *pre-dates* NLnet's
   current RFP system, where RFPs were submitted by EMAIL and there
   are manual records not fully integrated into the database.

also note: as the addition of sub-tasks *requires a change to the MOU*
it should NOT be taken lightly, i.e. should not be arbitrarily done
one by one, but only in "batches".

considerable care therefore has to be taken to ensure that NLnet are
not overloaded, nor that the EU Auditor is given grounds to become
"suspicious" because of a dozen or more alterations to the MOU.
and write your nickname (i.e. andrey etc.).

# Budget Estimation

Working out a time taken (and budget) for a sub-tasks requires
guestimating. A small self-contained task should take
approximately **1/2 a day up to 8 days (+/- 40%)**.

The total for a group of sub-tasks should be approximately
**5-25 days**. If a single tasks looks like it might take
longer than 8 days, it is **required** to break it into
smaller subtasks. Big tasks can quickly get out of hand, so
if in doubt, splitting a task is the better option.

Assume *1 month is appx EUR 3000* (this is an average; the value
may be higher depending on circumstances) and back-calculate.

These numbers come from Luke's
[comment #8 on bug #1126](https://bugs.libre-soc.org/show_bug.cgi?id=1126#c8)

Statistically speaking using the +/- 40% variance for each task,
and adding up over 20+ tasks will give a time estimate
**that is accurate to +/- 10%**.

*(Any sources on this?)*

However it is very important to have a *clear idea of what is
actually needed*, and to *not leave anything out*.

For example, when determining the task of adding instructions:

- For each instruction perform a thought experiment:
  "how many lines of HDL, how many unit tests?"
- Then from *- past experience -* estimate the total number
  of days.
- Assume 1 month is appx EUR 3000 and back-calculate.
- Put that number for each instruction (or group)
into comment 0, add them up, and make that the total
for the task.

*(Luke has used this method for the last 5 years based on 20 years
of project management, and it is **expected for the team to familiarise
themselves with it**)*

Also, make sure not to forget including **documentation** in your
estimate. This ensures a portion of grant money is allocated
to actually documenting the work involved.

Without documentation, it is not only difficult to teach newcomers about
the code in question, it makes it difficult to come back to the code
6-12 months later for maintenance and/or improvement
(not a rare situation in LibreSOC).

Don't forget to ask fellow project for help, they might be able
to help determine the scope of the work involved.

# "I'm thinking of doing... procedure"

## Preamble

Given the scale of this project and the critical reliance on certain parts
of it (such as devscripts, ISA csv files, ISACaller, etc.) on the work done
by the team, it is extremely important to raise any proposed changes and/or
improvements, and to wait for feedback _before_ implementing said changes.

Going forward, we all need to keep this in mind when working on
critical parts of the codebase.

To make good use of available time and budget, the LibreSOC team should
focus on:

1. Completing tasks under grant budget
2. Make small, incremental changes which keep the overall codebase functional.
3. When coming up with fixes or improvements which are intrusive to the
   _current_ workflow (which may slow the team down from completing tasks
   under grant budget), assign them to 'Future' milestone for grant
   applications going forward.

Why these three points?

1. Work that cannot be related to grant sub-tasks (even if indirectly, by
   bringing us closer to eventual completion), should be put aside _until
   future funding is secured/confirmed_.
2. Small changes make it easier and quicker to find mistakes. That's one of
   the reasons Luke has specified on [[/HDL_workflow]] to stick to small
   commits. *(Andrey: I need to improve on this myself)*
3. Big changes are inherently risky. When LibreSOC was just a few people
   (Luke and Jacob), it was easier to keep track of each other's progress.
   5 years down the line, the situation has changed.
   Keep in mind that changes to critical parts (whether big or small) will
   now affect at minimum Luke, Dmitry, Jacob, Sadoon, myself
   (perhaps also Cesar and Konstantinos, and so on).
   By going through the process of documenting a change in a new bug report,
   not only there's an opportunity to take a pause and think about
   repercussions, it also adds to the list of work for future grant
   applications (which will make it easier to draft focussed grants with
   realistic timescales and budget).

## Procedure

- If you discover a problem in code, raise a bug report, and use a
corresponding 'importance' setting depending on how serious you perceive the
issue to be. This will start a _discussion_.

**No work is to be started yet.**

- Based on generated discussion, determine if the issue is a *blocker to
current tasks under budget*. If it is a blocker , then the task 'importance'
to be set to 'major' or 'critical (or 'blocker').
*Andrey: need to clarify this*
If possible, a budget may be assigned after discussion and confirmation with
Luke and Andrey (depends on remaining budget/tasks).

- If the issue is *not a blocker*, but useful in future work, then the
'Future' milestone is to be assigned. The issue will be evaluated at a later
stage. At this point, no further time should be spent on this issue
(to prioritise outstanding tasks).

*(Andrey): Sometimes determining whether to use WONTFIX or INVALID is
difficult. Perhaps more examples would help?*

- If the issue is not a blocker, and the discussion shows that it is not
an issue at all, it is to be set to either of the following:
  - `RESOLVED DUPLICATE` - If the issue raised already exists.
    [Example, bug #962](https://bugs.libre-soc.org/show_bug.cgi?id=962)
  - `RESOLVED WONTFIX` - If the issue requires too much time or budget.
    [Example, bug #921](https://bugs.libre-soc.org/show_bug.cgi?id=921)
  - `RESOLVED INVALID` - If the issue does not align with project goals or
    methodology.
    [Example, bug #76](https://bugs.libre-soc.org/show_bug.cgi?id=76)
- The final status will be confirmed after *at least two other people* (other
  than the reporter) look at the bug report.
  For cases considered to be `WONTFIX` or `INVALID`, 48h should be given
  before the bug report is closed. This ensures the team has enough time to
  see the discussion before the issue disappears.

- Once the issue has been discussed and determined to be critical to current
grant sub-task/s, and budget considered, _then_ work can proceed in a separate
branch. Only after fixes have been confirmed to keep the CI tests passing,
can they be rebased (to keep commit history) into the master branch.

This procedure adds a time delay between the issue discovery and
start of work. This is important however because it allows for team members
to read bug updates without being overwhelmed and have time to add input.