BenchmarkDotNet v0.10.0

• Now BenchmarkDotNet is a part of .NET Foundation
• Job and Column API refactoring (see new documentation)
• Measurement engine improvements
• Horology enhancement (see TimeInterval and Frequency)
• Introduced RankColumn which is based on WelchTTest (see 157aabc3)
• JsonExporters refactoring (see the Exporters/Json section in the documentation)
  • Renamed JsonExporters classed and attributes
  • JsonExporters with custom settings
  • JsonExporters now includes information about the target type namespace (see #246).
• Add JetBrains.Annotations (see #253)
• RFC 4180 support in CSV exporters (see #241)
• Many bugfixes

Milestone details

In the v0.10.0 scope, 19 issues were resolved and 7 pull requests were merged. This release includes 85 commits by 7 contributors.

Resolved issues (19)

• #30 Better information when we haven't got a valid measurement
• #121 Strange # of "Launches" chosen with Count.Auto
• #154 PathTooLong exception on custom config
• #185 Report if difference between 2 benchmarks is statistically significance
• #241 .csv results output does not play well with Excel or Google Sheets
• #244 DefaultConfig StatisticColumn values
• #246 No namespace information?
• #265 Add ability to specify that benchmark requires STAThread
• #266 Don't assume that TargetType has reference to BenchmarkDotNet (assignee: @adamsitnik)
• #268 Print runtime of child processes in summary
• #271 Params attribute does not handle nullable types
• #272 [Setup] error when doing inheritance
• #276 System.EntryPointNotFoundException
• #280 Cannot run on OSX / Mono (System.Xml.XmlException: Root element is missing)
• #281 Results are exported twice for single run
• #288 IdleWarmup running off forever
• #291 [Bug] Incorrect results for targetCount:Auto
• #292 Support for Beta versions
• #296 [BUG] NRE in OutliersAnalyser

Merged pull requests (7)

• #253 Mark [Benchmark] as implying implicit use (by @roji)
• #267 Make shipped assemblies have CLSCompliant(true) applied (by @lahma)
• #277 Changed diagnosers flow, reduced heap allocations in Engine to 0 (by @adamsitnik)
• #278 Support Atlassian flavored wiki markup (by @lahma)
• #286 Mutable Job implementation (by @ig-sinicyn)
• #287 Fix docs: job API changed (by @ig-sinicyn)
• #293 Presenters: IFormattable support (by @ig-sinicyn)

Commits (85)

• a8b4e7 JsonExporters refactoring (by @AndreyAkinshin)
• e6a4ae JsonExporters: add information about namespaces, resolves #246 (by @AndreyAkinshin)
• 29ee0a Add Namespace column (by @AndreyAkinshin)
• 26c333 Better error message (by @arthrp)
• 12b313 Merge pull request #251 from arthrp/master (by @AndreyAkinshin)
• 03d00c Mark [Benchmark] as implying implicit use (by @roji)
• 394a93 Merge pull request #253 from roji/resharper-annotations (by @AndreyAkinshin)
• 4912ea Remove unnecessary specific shell reference (by @factormystic)
• 023115 Merge pull request #254 from factormystic/patch-1 (by @AndreyAkinshin)
• 972fee Big refactoring (by @AndreyAkinshin)
• cf839a Improved ranks (by @AndreyAkinshin)
• ef3ecf Extended TimeInterval and Frequency API (by @AndreyAkinshin)
• dfcc98 Minor fixes (by @AndreyAkinshin)
• fb3757 Improved CSV export, fixes #241 (by @AndreyAkinshin)
• f490d1 further root folder cleanup #228 (by @adamsitnik)
• f7a6a0 Added [MeansImplicitUse] and explanation to summary for SetupAttribute and Cl... (by @adamsitnik)
• 56b1f2 Configuration fix in JitOptimizationsTests (by @AndreyAkinshin)
• faac58 Introduce AnaylyzeLaunchVariance (by @AndreyAkinshin)
• 9953a9 Smart statistics in StatisticsColumnProvider (by @AndreyAkinshin)
• 2a6578 Minor fixes in docs (by @AndreyAkinshin)
• 03fb04 Introduce UnrollFactor (by @AndreyAkinshin)
• 137636 NewLine fix in OutputLogger (by @AndreyAkinshin)
• b35d52 Unique column support (by @AndreyAkinshin)
• 24e944 Don't assume that TargetType has reference to BenchmarkDotNet, fixes #266 (by @adamsitnik)
• 69330a Make shipped assemblies have CLSCompliant(true) applied (by @lahma)
• fb8402 Merge pull request #267 from lahma/features/cls-compliancy (by @adamsitnik)
• 0944b5 Make exported HTML valid, add alternating color to result table (by @lahma)
• 748a2c Merge pull request #269 from lahma/features/html-export-enhancements (by @adamsitnik)
• afff51 ExecutionValidator should not throw on overridden [Setup] methods, fixes #272 (by @adamsitnik)
• ab9db4 Support nullable types as [Params], fixes #271 (by @adamsitnik)
• ce4994 support for [STAThread], fixes #265 (by @adamsitnik)
• 13fa5c move ConsoleHandler to separate file for better readability (by @adamsitnik)
• 0e8e82 Print runtime of child processes in summary, fixes #268 (by @adamsitnik)
• 491a28 Print more info about runtime of child processes in summary (and in a nicer w... (by @adamsitnik)
• 0580a5 updated docs for #265 (by @adamsitnik)
• f9baa6 catch native exceptions when determining clock type, #276 (by @adamsitnik)
• 93a23d Check OS version in WindowsClock, fixed #276 (by @AndreyAkinshin)
• 8d65fe changed diagnosers flow, possibility to hook up before jitting, after setup a... (by @adamsitnik)
• f346ff added predefined attributes for diagnosers (by @adamsitnik)
• e7cc6b Support Atlassian flavored wiki markup (by @lahma)
• 376bd8 reduce memory allocated by Engine during run to increase MemoryDiagnoser accu... (by @adamsitnik)
• b5ab55 display results when runing when no diagnoser is attached (by @adamsitnik)
• 7ae2b5 Improved idle method for primitive types (by @AndreyAkinshin)
• e1213a Fix NRE in BuildJobRuntimes (by @AndreyAkinshin)
• 572fa4 Make code CLS-Compliant, fix CS3015 warning (by @AndreyAkinshin)
• 4070a1 preload all settings in ctors, introduced IEngineFactory (by @adamsitnik)
• 2f0df7 make Engine use InvocationCount (by @adamsitnik)
• 8a7a12 calculate Statistics without allocations! makes the code look bad, but saves ... (by @adamsitnik)
• d8fa70 remove last allocations from Engine.Run (by @adamsitnik)
• f2a106 perform Jitting after first Setup call, better naming, test fix (by @adamsitnik)
• be3ce3 introduce general catch with hopes to help with #280 (by @adamsitnik)
• 3c5e70 post code review changes (by @adamsitnik)
• 16e258 possibility to define custom Engine (by @adamsitnik)
• afa586 Merge pull request #277 from PerfDotNet/diagnosersFlow (by @adamsitnik)
• 534189 export files only once, not twice, fixes #281 (by @adamsitnik)
• a733b5 Mutable Job implementation (by @ig-sinicyn)
• 259647 Merge pull request #286 from ig-sinicyn/feature-mutable-characteristics (by @AndreyAkinshin)
• c184b8 Fix docs: job API changed (by @ig-sinicyn)
• 47b152 Merge pull request #287 from ig-sinicyn/fix-docs-jobs-updated (by @AndreyAkinshin)
• 868ffc Jobs: WithXxx() extension methods added back, docs updated (by @ig-sinicyn)
• eede54 Jobs: .With() methods now create new instances of the Job. (by @ig-sinicyn)
• 40f160 Test fixed (by @ig-sinicyn)
• fd92f7 Jobs: helper for .With() methods (by @ig-sinicyn)
• 307b72 Merge pull request #289 from ig-sinicyn/features-jobs-with (by @AndreyAkinshin)
• d576f5 Presenters: IFormattable support (by @ig-sinicyn)
• 8647e7 Fix #291 (by @AndreyAkinshin)
• 49c000 Merge pull request #293 from ig-sinicyn/fix-presenter-culture (by @AndreyAkinshin)
• 684334 Merge pull request #278 from lahma/features/atlassian-wiki-markup (by @AndreyAkinshin)
• 7f3ca5 Make BenchmarkRunnerCore.Run public (by @AndreyAkinshin)
• 10a91e Allow using newer versions of Microsoft.NETCore.App, fixes #292 (by @adamsitnik)
• 134d74 Misc improvements in Exporters (by @AndreyAkinshin)
• 33c63f Respect RemoveOutliers in the RunResults (by @AndreyAkinshin)
• 5e022b Fix #291, part 2 (by @AndreyAkinshin)
• 4b6ea9 Improved name for the Id CharacteristicColumn (by @AndreyAkinshin)
• af6f8a Fix some compilation warnings (by @AndreyAkinshin)
• 2f8584 Analysers refactoring + OutliersAnalyser (by @AndreyAkinshin)
• b11935 Make BenchmarkRunnerCore public (by @AndreyAkinshin)
• 781740 Specify generated id for the default job (by @AndreyAkinshin)
• 846255 JobTests.Test01Create fix (by @AndreyAkinshin)
• 9f9544 Check if there is no MainTarget measurements in OutliersAnalyser, fixes #296 (by @AndreyAkinshin)
• 503b04 Proper exception in GetStatistics for empty input, see #296 (by @AndreyAkinshin)
• c667aa Fix path to logo (by @AndreyAkinshin)
• 0b91c3 Fix null check in JobMode.ApplyCore (by @AndreyAkinshin)
• 2edb56 Copyrights and links update (by @AndreyAkinshin)
• 2edb9a Set library version: 0.10.0 (by @AndreyAkinshin)

Contributors (7)

• Adam Sitnik (@adamsitnik)
• Andrey Akinshin (@AndreyAkinshin)
• Arthur (@arthrp)
• factormystic (@factormystic)
• ig-sinicyn (@ig-sinicyn)
• Marko Lahma (@lahma)
• Shay Rojansky (@roji)

Thank you very much!

Additional details

Date: November 10, 2016

Milestone: v0.10.0

NuGet Packages:

• https://www.nuget.org/packages/BenchmarkDotNet/0.10.0
• https://www.nuget.org/packages/BenchmarkDotNet.Core/0.10.0
• https://www.nuget.org/packages/BenchmarkDotNet.Toolchains.Roslyn/0.10.0
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.Windows/0.10.0

Online Documentation: https://dotnet.github.io/BenchmarkDotNet/