> Scientists really need to publish their code artifacts, and we can no longer just say "Well they're scientists or mathematicians" and allow that as an excuse for terrible code with no testing specs.
You are blaming scientists but speaking from my personal experience as a computational scientist, this exists because there are few structures in place that incentivize strong programming practices.
* Funding agencies do not provide support for verification and validation of scientific software (typically)
* Few journals require assess code reproducibility and few require public code (few require even public data)
* There are few funded studies to reproduce major existing studies
Until these structural challenges are addressed, scientists will not have sufficient incentive to change their behavior.
> Scientific code needs to have tests, a minimal amount of test coverage, and code/data used really need to be published and run by volunteers/editors in the same way papers are reviewed, even for non-computer science journals.
Second this. Research code is already hard, and with misaligned incentives from the funding agencies and grad school pipelines, it's an uphill battle. Not to mention that professors with an outdated mindset might discourage graduate students from committing too much time to work on scientific code. "We are scientists, not programmers. Coding doesn't advance your career" is often an excuse for that.
In my opinion, enforcing standards without addressing this root cause is not gonna fix the problem. Worse, students and early career researchers will bear the brunt of increased workload and code compliance requirements from journals. Big, well-funded labs that can afford a research engineer position is gonna have an edge over small labs that cannot do so.
You are blaming scientists but speaking from my personal experience as a computational scientist, this exists because there are few structures in place that incentivize strong programming practices.
* Funding agencies do not provide support for verification and validation of scientific software (typically)
* Few journals require assess code reproducibility and few require public code (few require even public data)
* There are few funded studies to reproduce major existing studies
Until these structural challenges are addressed, scientists will not have sufficient incentive to change their behavior.
> Scientific code needs to have tests, a minimal amount of test coverage, and code/data used really need to be published and run by volunteers/editors in the same way papers are reviewed, even for non-computer science journals.
I completely agree.