Validating and documenting migration files

[dachary]

Bonjour,

When restore-repo reads YAML file that were previously created by dump-repo, it would be helpful to sanity check their content.

It would also make is easier for third party software (other forges or any other) to create such files for Gitea to ingest. Or use files dumped by Gitea for other purposes. The schema used for validation can also include a documentation and it could grow into a de-facto file format.

To illustrate this, a diff implements this idea for issues only but it would be trivial to similarly cover other data structures.

• The –validation option is added to restore-repo
• JSON schema describes issues, labels and reactions
• A helper loads the YAML dump file and optionally (if --validation is set) validates its content against the corresponding schema.
• The helper is used in RestoreRepo and helps keep it DRY
• The schemas are added in a bindata file to be used in the binary
• Tests verify validation border cases and error conditions

How do people feel about going in this direction?

[lunny]

Because I think it’s not very stable of the file format. If we want to share it with others, we need to make a standard at first.

[dachary]

Right: at the moment it is an internal format that is not designed or intended to be shared. Adding validation is a baby step that is both useful immediately and helps create a standard to be shared with others later on.

A use case where it can be immediately useful is when someone dump-repo a project with gitea 14.2 and a much later tries to restore-repo the same project using gitea 18.3. If the format changed, validation will display an error instead of trying to ingest files that will be interpreted incorrectly.

@lunny

If we want to share it with others, we need to make a standard at first.

The standard internal to Gitea is defined by these files issues, labels, reactions etc. It can then become a standard shared outside of Gitea.

Does that make sense?

[lunny]

Maybe you could send a PR back to Gitea.

[dachary]

For archive (discussion in chat) a related PR:

Escape string and blob results from dump more correctly

[lunny]

I have reviewed that and will merge it after CI PASS.

[dachary]

The pull request was submitted by @realaravinth and passes CI. Thanks a lot for your review