Recent blog posts on Haiku Projecthttps://www.haiku-os.org/blog/en-USHaiku, Inc.(C) 2001-2024 Haiku, Inc. All rights reserved.Thu, 14 Mar 2024 10:00:00 -0400Haiku Activity & Contract Report, February 2024https://www.haiku-os.org/blog/waddlesplash/2024-03-14_haiku_activity_contract_report_february_2024/waddlesplashThu, 14 Mar 2024 10:00:00 -0400https://www.haiku-os.org/blog/waddlesplash/2024-03-14_haiku_activity_contract_report_february_2024/<p>This report covers hrev57561 through hrev57615.</p> <h3 id="applications">Applications</h3> <p>PulkoMandy fixed some UI issues in Screen preferences, including invalid Hz being displayed and a confusing translation string.</p> <p>madmax fixed updating fields in the FileTypes &ldquo;Application type&rdquo; window, fixing a rather old bug.</p> <p>bitigchi made some improvements to UI strings in Terminal (use of a Unicode multiplication sign instead of &ldquo;x&rdquo;), and added a &ldquo;Make a donation&rdquo; link to AboutSystem.</p> <p>apl fixed a compiler warning in HaikuDepot.</p> <p>humdinger made Tracker offer &ldquo;Get info&rdquo; as an option in the &ldquo;Broken link&rdquo; alert dialog.</p> <p>humdinger fixed Installer and AboutSystem to pick light/dark versions of the main logo as appropriate.</p> <p>OscarL altered some shortcuts in Terminal to try and allow Shift+{Arrow,Home,End} to be used in applications running inside Terminal. (However, this seems to have caused regressions in other areas, and a followup change is in the works.)</p> <p>nephele fixed the text color in DeskCalc&rsquo;s expression view.</p> <p>Calisto-Mathias fixed an alignment issue with the &ldquo;Close window&rdquo; checkboxes in MediaPlayer&rsquo;s settings. They also removed the now-unnecessary tab view from the Devices app.</p> <h3 id="command-line-tools">Command line tools</h3> <p>korli fixed <code>listusb</code>&rsquo;s display of maximum packet sizes, added display of the <code>length</code> descriptor field, and fixed a crash which happened when printing information of some USB audio devices.</p> <p>korli (after adding some necessary methods to <code>libbsd</code>) updated the <code>ping</code> command to the current version from FreeBSD, fixing a number of bugs and adding new features. waddlesplash followed this up with a similar upgrade to the <code>traceroute</code> command.</p> <h3 id="kits">Kits</h3> <p>trungnt2910 gave the layout spacing enum a name, so it can be more easily used in API bindings to other languages (e.g. .NET.)</p> <p>PulkoMandy reworked some code in the FFmpeg plugin and added some code to support newer versions of FFmpeg.</p> <p>ilzu contributed a fix to <code>BKeyStore</code> to make its behavior match the documentation.</p> <p>korli fixed BMenu&rsquo;s <code>SortItems</code> routine, which had been broken after switching to using <code>std::stable_sort</code> internally.</p> <p>waddlesplash fixed the &ldquo;raw&rdquo; media decoder to properly pass errors back up the chain (so that <code>B_LAST_BUFFER_ERROR</code> will actually be seen by applications.)</p> <p>waddlesplash fixed the FFmpeg plugin&rsquo;s frame-count computation to be more accurate, fixing BePac Deluxe (a closed-source BeOS game) once more (it checks frame count values in its audio files are exactly correct, and errors out if they&rsquo;re not.)</p> <p>nephele added a <code>Brightness</code> API to <code>rgb_color</code> (replacing the old <code>perceptual_brightness</code> standalone function), and added <code>IsLight</code>, <code>IsDark</code>, and <code>Contrast</code> utility functions to go with it. He also wrote documentation for these new APIs and applied them across the tree in places where they made sense to use.</p> <p>waddlesplash fixed the &ldquo;multi audio&rdquo; media add-on (the main one used for communication with audio hardware drivers) to actually use the reported record channel count rather than assuming 2, fixing audio input for USB headsets.</p> <p>PulkoMandy fixed a shadowed variable problem in <code>BSpinner</code>, fixing the values sent in the &ldquo;invoke&rdquo; messages.</p> <p>nhtello made a number of improvements to styles and visibility of the &ldquo;Flat&rdquo; decorator in dark mode.</p> <p>X512 fixed the build of the UVC webcam driver on 64-bit (however, it&rsquo;s missing a bunch of code necessary for it to actually be used with most webcams.)</p> <p>bitigchi added support for setting the output precision of <code>BNumberFormat</code>.</p> <h3 id="servers">Servers</h3> <p>PulkoMandy turned off some unnecessary logging in the package server.</p> <p>bitigchi added automatic percentage string formatting to <code>notification_server</code>.</p> <p>korli switched the default UI font to &ldquo;Noto Sans&rdquo;. Previously it was &ldquo;Noto Sans Display&rdquo;, but the Noto fonts project is discontinuing this variant, and &ldquo;Noto Sans&rdquo; itself is not too much different. madmax updated the &ldquo;Symbols&rdquo; font name as well.</p> <p>X512 fixed a use-after-free on deinitialization in <code>Screen</code> and a memory leak in <code>BitmapManager</code> inside <code>app_server</code>.</p> <h3 id="drivers">Drivers</h3> <p>OscarL fixed the build of the <code>acpi_ac</code> driver, which detects if the system is connected to AC power. He also submitted a fix for some typos in log messages in the <code>usb_rndis</code> driver, fixed SMAP violations in <code>acpi_ac</code>, <code>acpi_lid</code>, and <code>acpi_thermal</code>, and added some sanity checks to <code>acpi_ac</code> and <code>acpi_lid</code>.</p> <p>waddlesplash fixed a compiler warning in the usb_audio driver, and disabled USB 2.0 audio device support for now (as it does not work and causes problems.)</p> <p>waddlesplash made loopback (and tunnel) devices' packets not have ethernet headers. Originally they never did; then, when packet capture was fixed for loopback, dummy ethernet headers were added; but after some suggestions from one of the <code>libpcap</code> developers, changes were made to <code>libpcap</code>&rsquo;s Haiku support to make the dummy ethernet headers unnecessary, and so they were removed once more, simplifying things.</p> <p>korli implemented <code>connect</code> for IPv4 and IPv6 raw sockets (and removed a workaround in the new <code>ping</code> command for this.)</p> <p>X512 changed PCI MSI vector numbers to always be 32 bits, and adjusted all drivers using MSIs to match.</p> <p>waddlesplash ported a small fix to the <code>idualwifi7260</code> and <code>iaxwifi200</code> drivers from OpenBSD.</p> <h3 id="file-systems">File systems</h3> <p>InfiniteVerma contributed a <code>touch</code> command implementation to the <code>fs_shell</code>.</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>X512 fixed the &ldquo;DPC&rdquo; (deferred procedure call) module to not cause problems when passed the same procedure to call multiple times in a row.</p> <p>sunfishcode contributed a change to make Haiku&rsquo;s <code>AT_FDCWD</code> have a value of <code>-100</code> (much like other OSes) instead of <code>-1</code>, to allow for error values to be more clearly distinguished from it. (For the moment, the kernel accepts either value, so old applications won&rsquo;t break immediately; but in the future, this may be changed. BeOS did not have <code>AT_FDCWD</code>.)</p> <p>korli made a fix to the device_manager to allow the virtio-GPU driver to actually be loaded properly.</p> <p>waddlesplash fixed the virtual memory manager&rsquo;s &ldquo;reserve memory&rdquo; routine to return immediately without waiting when the amount requested to be reserved is greater than the total amount available. This fixes some stuttering in Falkon (Chromium) on systems with not-so-large amounts of memory.</p> <h3 id="build-system">Build system</h3> <p>kallisti5 bumped the system ICU version to 74 (and rebuilt all packages that depend on ICU as well, so only ICU 74 is installed with the base system now.) waddlesplash made a followup fix to libroot&rsquo;s locale code that had been broken in the upgrade. (This is one of the things that needed to be done before beta5.)</p> <p>nephele fixed the compilation of some tests, and deactivated a few others from the build.</p> <h3 id="documentation">Documentation</h3> <p>InfiniteVerma contributed some minor improvements to the internals documentation on filesystems.</p> <p>PulkoMandy added mention of <code>libnetwork</code> on the page about <code>libroot</code> in the Haiku Book, and added a &ldquo;Storage Kit Introduction&rdquo; page which gives basic details on the filesystem hierarchy.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>Nothing of note that was ARM or RISC-V specific in February&hellip; but there&rsquo;s already been some activity in March, so stay tuned.</p> <h3 id="are-we-beta5-yet">Are we beta5 yet?</h3> <p>A few more tickets in the milestone were fixed, including the &ldquo;ICU upgrade&rdquo; one, but a few were also added (some migrated from HaikuPorts that turned out to be regressions in Haiku or its buildtools, etc.)</p> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p>So you want to help with WebKit?https://www.haiku-os.org/blog/pulkomandy/2024-02-28_so_you_want_to_help_with_webkit/pulkomandyFri, 01 Mar 2024 12:00:00 -0500https://www.haiku-os.org/blog/pulkomandy/2024-02-28_so_you_want_to_help_with_webkit/<div class="alert alert-info"> This blog post was originally <a href="https://discuss.haiku-os.org/t/my-progress-on-webkit2-port/11653/64">a forum post</a>. It is reproduced here on the website to make it easier to find and reference. </div> <p>I heard that some more people may be interesting in helping with WebKit. So here is a summary of the current state, the things I think need work, or the possible future paths to explore.</p> <h3 id="keeping-webkitlegacy-up-and-running">Keeping WebKitLegacy up and running</h3> <p>The Web moves fast these days. So we have to stay very up to date with upstream WebKit. Until we have a nice and shiny WebKit2 browser, and, anyways, even after that, we need to keep things up to date.</p> <p>Currently I take care of this myself. The reason is, it involves merging changes from upstream with our branch, and that’s pretty much impossible to review as a Github pull request (it would show everything that’s changed in the upstream WebKit code, which is usually a lot).</p> <p>Anyways, here’s how I do it:</p> <ul> <li>Get the latest commits from upstream webkit</li> <li>Looks for commits with a message saying “Versioning.” in the main branch. These are internal commits from Apple changing some version numbers, but I found that these are usually stable versions of WebKit that will at least compile without too much problem</li> <li>Pick the next such commit, merge it with our branch (using “git merge”).</li> <li>Try to build, fix any issues that show up</li> <li>Make sure the browser still runs.</li> </ul> <p>There are usually some function that have changed prototype and the Haiku implementation needs to be updated, and the occasional merge conflict in one of the few files that have specific code for multiple platforms (typically we just add <code>|| OS(HAIKU)</code> to existing cases, but there are a few places where we have to do our own thing.</p> <p>This brings me to the next two topics…</p> <h3 id="upstreaming-our-changes">Upstreaming our changes</h3> <p>A lot of Haiku specific code is maintained in our fork of the WebKit repository. The reason for this is I never took the time to upstream all these things. There are no easily extracted commits (because the only way to keep up with upstream is to use merge instead of rebase), and as a result, the relevant changes have to be manually re-done as separate, nice and clean commits that upstream will accept (hopefully).</p> <p>The process is something like this:</p> <ul> <li>Do a “git diff” between our fork and the last upstream commit we merged</li> <li>Look at the differences, find something that can easily be upstreamed</li> <li>Create a nice patch and submit it to WebKit bugtracker or GitHub (there is an helper script in the repository to prepare and submit a patch)</li> <li>Follow up on feedback from code review</li> <li>If there is no one reviewing the code, I found that complaining about it on social networks might work (because some WebKit devs are watching my social network account, I guess). There are maybe more reasonable ways.</li> </ul> <p>Where to start with the upstreaming changes: probably WebCore, WTF and JavaScriptCore. If we start submitting anything substantial, the WebKit upstream will require us to provide a buildbot to build our code. That is then used to build patches submitted to their bugzilla or github, and verify that the code does not break our build.</p> <p>Until we get this working, we can still submit changes that fix cross-platform code (there are a handful of these in our fork), or that just add Haiku to the list of OS using a specific codepath. Which are also the ones more likely to cause problem when merging upstream changes, so that would still be very helpful.</p> <p>Merging the code in the WebKitLegacyDirectory is probably not going to happen since upstream is slowly removing that directory (only the iOS port still has it, to run very old iOS apps that need the legacy API).</p> <h3 id="fixing-and-improving-the-test-suite">Fixing and improving the test suite</h3> <p>This is a bit more complex work and requires in-depth investigation of problems.</p> <p>WebKit comes with several thounsands of tests. Normally, every bug fix or change should be submitted with a corresponding test.</p> <p>These are non-regression tests, and, in the cross platform and configurable nature of WebKit, there is no hope that all tests will ever run on all platforms and configurations. So, the idea is that there is a “TestExpectations” file for each platform, that can define which tests to skip (because the feature is not implemented and there’s no point), which are expected to fail, which are expected to pass. After making a change, a developer can then run the tests and see if anything was broken or fixed. The bots on WebKit build infrastructure also check for this, and when someone submit a patch, they will warn if tests were broken (or fixed without updating the TestExpectations files).</p> <p>The testsuite used to run on Haiku, but last time I tried it it completely crashed my machine (probably an app_server crash), and also I ran into problems with Debugger. The testsuite runs one process per core, and configures debug_server to save debug reports if any of them crash. However, Debugger gets very confused if we ask it to save multiple debug reports at the same time. As a result, we end up with several instances of Debugger trying to save debug reports, and eventually the system runs out of memory. Fixing this would be great, because then, the testsuite would have more chances to run.</p> <h3 id="experimenting-with-cross-platform-code">Experimenting with cross-platform code</h3> <p>The current implementation of WebKit for Haiku tries very hard to use the native APIs for everything. We have scaled back for now on the HTTP implementation, and are using cURL for that (it fixed several bugs compared to our own HTTP library). But for everything else, and in particular for all graphic rendering, we use offscreen BBitmap and BView.</p> <p>It would be interesting to experiment with using cairo. This can easily be done beind a compile time option (mainly disabling the Haiku specific drawing code when the Cairo one is enabled). There is some integration needed so that in the end, cairo renders to a BBitmap that can be send on screen as usual.</p> <p>It would be interesting to have this as an option, even if only to compare the rendering and decide if bugs are on WebKit side or in our platform specific code. But it would also fix several problems/limitations of the current drawing code: lack of support for shadows, for font ligatures, for some drawing compositing modes, etc.</p> <h3 id="the-web-inspector">The web inspector</h3> <p>WebKit codes with a web inspector. This is mostly implemented as an html/javascript app, that interfaces with the WebKit core and allows to inspect the content of pages. Surely this would be helpful to debug some of the rendering issues. It is “almost completely” implemented, I am not sure what is missing, but I think not a lot.</p> <p>There is also a “remote inspector” that allows to connect from another machine (used on the iOS version, because running the web inspector on an iPhone is no fun, and similarly in game consoles, embedded systems, …), maybe that is worth exploring too but I have not researched it a lot.</p> <h3 id="working-on-webkit2">Working on WebKit2</h3> <p>WebKit2 is a new interface to the WebKit engine that runs several parts of the code in different processes. There is at minimum a “web” process that does the html/css/javascript, an “UI” process that does the display (that’s the actual web browser), and a “network” process that does, well, all the network things.</p> <p>This allows the web browser to be a lot less complicated and just communicate with the other processes. It also means the browser is unlikely to completely crash, instead, most likely the web process will crash, which does not imply losing your list of open tabs, non-saved cookies, etc. It also allows to run multiple web processes, for example, one per browser tab, so the whole browser is not completely frozen when one tab crashes.</p> <p>Most of the work to get this built and a simple testing browser on top of it has already been done (by Rajagopalan Gandhagaran in Google Summer of Code). However, the code needed a lot of cleanup, and is now several years behind. I have a branch that I rebase every few months with an up to date but non-working version of this code. The main problem is the way the different processes establish connections between each other. Basically the expectation is that the web process creates a socketpair() (similar to a pipe(), but with UDP-like datagram behavior), and then passes each end of the socketpair to the two other processes, and the connection between the UI and network process is establised in this way. Passing file descriptor to the other processes is possible, but in the current attempt to implement WebKit2, we had decided to use BLooper/BMessage instead. So, it would mean that the web process needs to create a BMessenger targetting a BLooper in another app that it knows nothing about (and the other app may even not be running yet). This seems just about possible, but we may need some new APIs here or just do some workaround/hack instead.</p> <p>Unfortunately, I am busy with many other things and I have been unable to work at this problem long enough to fit it all in my mind and find a solution. I can only look at it every few months, and by the time I have reloaded all the info in my mind, I run out of time again. So, help on investigating this and getting it up and running would be great.</p> <p>There is also another option here: as mentioned, the other platforms use a socketpair(). We have an implementation of that now, so we could just as well stop trying to push BMessage beyond what they are supposed to do, and use socketpairs like everyone else. This would allow to run WebKit2 with a lot less platform specific code to write, since we could use the “Unix” variant of most stuff (MainProcessUnix, etc) instead of having to write our own. However, it creaet the possible issue that the main thread will not be a BApplication, and, at least for the UI Process (the web browser, that needs to open BWindows to interact with the user), that might be a problem. Or maybe it will work just fine, I’m not really sure.</p> <h3 id="continuing-the-work-to-merge-the-webkitlegacy-and-webkit2-branches">Continuing the work to merge the webkitlegacy and webkit2 branches</h3> <p>There is normally no conflict between WebKit2 and WebKitLegacy. They can be enabled separately or together, and the cmake build will take care of configuring and building everything. However, our current WebKit2 branch also has a number of hacks and work-in-progress things in shared directories (for example, adding a lot of tracing in WebCore classes around the process initialization). It would be great to get all this code cleaned, rebased, and ready for merging into the WebKitLegacy branch. This way we would have only one branch to maintain, instead of two, making both the upstream merging/rebasing and the submission of patches to upstream a lot easier.</p> <p>The indentation is also all over the place (in both branches, really) due to Haiku and WebKit not agreeing on the Great War of Space vs Tabs, and I think the decision that all .h files for WebKitLegacy providing the public API that Haiku apps can use would follow Haiku coding guidelines, while everything else would follow WebKit style. This is a bit confusing and hard to maintain.</p> <p>Certainly not the most exciting work: rebasing changes, cleaning indentation, cleaning commit history and removing debug traces or splitting them to separate commits. But, it would be very helpful.</p> <p>I hope this gives a good overview of the needed work, and also that a lot of it is caused by decisions and problems on our side, and not at all the fault of WebKit upstream. I also hope it inspires more people in having a go at it and see if they can fix one or two small things.</p> <p>In any case, thanks for reading this to the end! 🙂</p>Haiku Activity & Contract Report, January 2024 (ft. USB audio)https://www.haiku-os.org/blog/waddlesplash/2024-02-13_haiku_activity_contract_report_january_2024/waddlesplashTue, 13 Feb 2024 04:30:00 -0400https://www.haiku-os.org/blog/waddlesplash/2024-02-13_haiku_activity_contract_report_january_2024/<p>This report covers hrev57494 through hrev57560.</p> <h3 id="usb-audio">USB audio</h3> <p>The most notable set of changes last month was a series of fixes to the XHCI (USB3) driver and then the USB audio driver itself, culminating in the USB audio driver being turned on by default for nightly builds!</p> <p>A few weeks back, X512 noticed an oversight in the XHCI driver&rsquo;s handling of &ldquo;isochronous&rdquo; transfers (which pretty much only USB media devices like audio or webcams use.) I refactored the relevant portions of the driver to correct the oversight, as well as making a tweak to how the USB audio driver schedules transfers to be more in line with what other OSes do, after which the USB audio driver started working pretty much immediately. (It had originally been written back in the days of USB 1.0 controllers and worked to some degree then, and in the years since was repeatedly patched to keep it in line with current kernel APIs and to fix bugs, crashes, etc.; without that maintenance, it likely wouldn&rsquo;t have worked so smoothly on the first try.)</p> <p>A few more rounds of cleanup and bugfixing later to the XHCI driver (more robust error reporting, fewer messages printed to syslog, etc.), the USB stack (moving some functionality from the bus drivers into the stack, adjusting returned error codes across bus drivers for consistency), and the USB audio driver (adjusting timeouts, increasing some buffer sizes, improving error reporting), things seemed to be working stably enough that it seemed safe to turn it on in the nightly builds, and so it was (in the last commit of the month, in fact.)</p> <p>There&rsquo;s still a few caveats (e.g. only USB 1.0 audio devices are supported, rather than 2.0 devices, and they may not work when connected to anything but USB 3 controllers), and also some <code>media_server</code> bugs which mean switching your outputs from an internal audio card to a USB device is a bit finicky. There&rsquo;s a <a href="https://discuss.haiku-os.org/t/usb-audio-enabled-in-nightly-builds/14576/16">forum thread</a> with some details and discussion, if you want to try it out.</p> <h3 id="applications">Applications</h3> <p>jscipione fixed crashes in Icon-O-Matic caused by the auto-scroll changes in BListView.</p> <p>jscipione adjusted Keymap to put the status icons in the &ldquo;Modifier keys&rdquo; window inside the menu-fields. (However, there have been some reports the icons have stopped showing up after this change, so more work may still be needed.)</p> <p>korli added detection of &ldquo;Not charging&rdquo; battery states to PowerStatus.</p> <p>apl did some code cleanup and fixed some compiler warnings in HaikuDepot. He also refactored the parts of the code that fetch and display screenshots.</p> <p>humdinger moved the &ldquo;TV&rdquo; app into the <code>haiku_extras</code> package. It&rsquo;s only useful with real TV tuner devices, which aren&rsquo;t especially common these days, and which Haiku doesn&rsquo;t have too many drivers for, either. (While at it, he also added a description to the <code>extras</code> package info, and jmairboeck later added proper <code>provides</code> declarations for TV and another tool in the package.)</p> <p>jscipione removed the remnants of Trash-related settings from Tracker, and cleaned up the code that had checked them. He also performed some other miscellaneous code cleanup throughout Tracker.</p> <p>waddlesplash added logic to PowerStatus to allow it to be automatically installed in Deskbar on first boot, but only if there&rsquo;s actually a battery detected.</p> <h3 id="command-line-tools">Command line tools</h3> <p>madmax fixed some compiler warnings in the <code>elfsymbolpatcher</code> tool.</p> <p>waddlesplash fixed the <code>desklink</code> custom-item system to check what size Deskbar tray entries actually are, and use that, rather than a default (small) size.</p> <h3 id="kits">Kits</h3> <p>PulkoMandy updated the FFmpeg media add-on with some initial changes for FFmpeg 6 support (mostly disabled behind preprocessor flags.)</p> <p>korli fixed a positioning problem in <code>BAdapterIO</code> (a <code>BDataIO</code> implementation used in the Media Kit and in some applications for playing media streamed from the internet.) He also added a <code>FlushBefore()</code> method to the API, to allow unused/unneeded data to be discarded.</p> <p>humdinger made a quick-fix in the <code>registrar</code> for a problem with the &ldquo;supporting applications&rdquo; functionality: it didn&rsquo;t work for MIME types that had upper-case characters in them. waddlesplash came by later on and cleaned up and fixed the underlying problems in the supporting-apps classes, and undid the quick-fix in the <code>registrar</code>.</p> <p>PulkoMandy adjusted BMenu to use <code>std::stable_sort</code> internally in the implementation of <code>SortItems</code>, so that successive sorts don&rsquo;t swap the order of items that compare equally.</p> <p>madmax fixed some bugs in <code>BOutlineListView::ItemUnderAt</code> and adjusted it to accept <code>NULL</code> to mean the parent of the topmost items. (He adjusted the documentation and testsuite at the same time.)</p> <p>waddlesplash tried to fix a regression in BScrollView that was causing &ldquo;blank space&rdquo; to appear in some applications. (The fix solved problems in some applications, but not others, it seems.)</p> <h3 id="servers">Servers</h3> <p>X512 reduced the minimum timer interval permitted by the <code>registrar</code>, which is used to implement <code>BMessageRunner</code>. (This may be useful for developers who want to use <code>BMessageRunner</code> to implement animations, or things like that.)</p> <p>korli modified app_server to scan recursively in <code>/dev/graphics</code> for devices.</p> <p>madmax removed some obsolete code from <code>runtime_loader</code>.</p> <p>waddlesplash fixed the <code>registrar</code> to truncate the recents (apps, files, etc.) lists to 100 items maximum, rather than letting them grow indefinitely.</p> <h3 id="drivers">Drivers</h3> <p>waddlesplash fixed some SMAP violation KDLs in the <code>mmc</code> and <code>nvme</code> disk drivers, which occurred when running the <code>driveinfo</code> command on them.</p> <p>PulkoMandy adjusted the PCI subsystem to handle &ldquo;non-fixed&rdquo; hardware addresses (i.e. addresses with some values unset which have to be computed.) This was done as part of work to fix boot-failure KDLs that happen on some hardware, but it seems there&rsquo;s still more to be done here.</p> <p>waddlesplash refactored parts of the PCI subsystem to be able to report an arbitrary number of memory ranges, rather than being hard-coded to a maximum of 6. (At present, the memory ranges in question aren&rsquo;t actually used anywhere yet, so this likely won&rsquo;t fix any problems.)</p> <p>waddlesplash synchronized the <code>idualwifi7260</code> and <code>iaxwifi200</code> drivers, plus the <code>net80211</code> stack they use, with OpenBSD upstream.</p> <p>korli added some more device IDs to the AMD Cryptographic Coprocessor driver (an RNG source.)</p> <p>korli added a modesetting driver for <code>virtio_gpu</code> devices, and added it to the standard builds. These are the ones created by QEMU when <code>-device virtio-vga</code> is specified.</p> <p>waddlesplash fixed the XHCI driver to properly handle &ldquo;stopped, length invalid&rdquo; error events, plus a few other problems. This will reduce syslog spam and improve error recovery under a number of conditions. He also refactored how it &ldquo;links&rdquo; USB transfers into endpoint rings to avoid &ldquo;double-links&rdquo;, which might fix strange behavior seen on some controllers.</p> <p>waddlesplash fixed a use-after-invalidate bug (which was probably rarely triggered, if ever) in the IPv4 kernel module, and restructured the code to prevent similar problems from happening in the future. He also fixed the module to clamp the overall MTU as needed, in case a lower layer returned a value greater than the maximum size of an IPv4 packet.</p> <p>waddlesplash fixed the packet-input routine in the FreeBSD compatibility layer to handle being passed multiple packets at once, same as in FreeBSD. At present, the only code imported into Haiku which makes use of this functionality is <code>iflib</code> (used by <code>ipro1000</code> and <code>intel22x</code>), which had a workaround added in Haiku to not need it; but it seems the workaround didn&rsquo;t work quite right and could cause memory leaks, as removing the workaround after implementing the functionality in the compatibility layer seemed to solve some leaks.</p> <p>waddlesplash added code to the TCP module inside the <code>send()</code> implementation to ensure data is actually being sent before sleeping to wait for more space in the send queue to be available. (This may fix traffic stalls in some applications that send more data at once than can fit in the TCP send queue, and also do not specify the <code>NOWAIT</code> flag to return immediately when the queue is full.) He then removed the overridden default buffer size (32KiB) the TCP module applied to send queues, meaning the system default will now be applied (64KiB), but also values specified by applications might have a chance to get used now.</p> <p>waddlesplash added some code to the USB disk driver to cancel more types of queued transfers, in an effort to fix some rare KDLs. (It doesn&rsquo;t seem to have fixed the problem, unfortunately.)</p> <p>waddlesplash did some refactoring in the TCP module to the sending logic, breaking some larger functions up into a set of smaller ones; mostly a non-functional change, but making the code easier to follow and modify.</p> <p>waddlesplash fixed the TTY layer to return success (instead of failure) on partial writes, so that the calling application will know that some bytes were actually written. This fixes problems with pasting large amounts of data into shell sessions over SSH with some applications.</p> <p>korli added some better handling of DDI and some code for Gen12 to the <code>intel_extreme</code> modesetting driver.</p> <h3 id="file-systems">File systems</h3> <p>waddlesplash fixed some bugs in <code>userlandfs</code> that were preventing most FUSE filesystems from working properly; however, in the process, he had to disable the (new since beta4) caching support for FUSE, which seemed to not be working properly, and instead causing errors to spuriously be returned. He also adjusted a memory allocation in the kernel userlandfs module to fix/prevent some SMAP violation KDLs that happened under certain circumstances.</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>PulkoMandy fixed <code>features.h</code> (a header implicitly included to detect what features should be exposed in other, mostly POSIX, headers) to still work when the BSD compatibility headers (<code>libbsd</code>, not related to the kernel FreeBSD compatibility layer) are not in the compiler&rsquo;s include path, which is now needed when building Haiku itself.</p> <p>X512 fixed a file descriptor leak inside the kernel implementation of the <code>preallocate</code> function. This had been the culprit causing memory leaks in some ported web browsers (Falkon and Web among them), which were really leaks of shared-memory files on the <code>ramfs</code>.</p> <p>Following X512&rsquo;s change, waddlesplash went through and cleaned up file descriptor and vnode management in the kernel VFS layer, switching as many places as possible to use <code>FileDescriptorPutter</code>s and <code>VnodePutter</code>s (and replacing some variant implementations of those classes) rather than manually invoking <code>put_vnode</code>/<code>put_fd</code> (as, if these had been used throughout, the bug causing the leak wouldn&rsquo;t have happened at all.) This also eliminated some of the uses of <code>goto</code> in the VFS subsystem. He also refactored some functions that previously took <code>vnode**</code> parameters to instead take <code>VnodePutter&amp;</code>s, to simplify code and prevent memory leaks.</p> <p>korli fixed <code>create_vnode</code> to also create non-existent entries when traversing. (This fixes <code>open()</code> failing when passed a path which is a symbolic link to a non-existent file.)</p> <p>waddlesplash changed the kernel virtual memory subsystem to avoid committing memory inside <code>mmap</code> for private mappings (i.e. those without <code>MAP_SHARED</code>) which are non-writable (i.e. without <code>PROT_WRITE</code> specified). Instead, memory will be committed inside <code>set_area_protection</code> or <code>set_memory_protection</code> (<code>mprotect</code>), when/if the area or portions of it are made writable. This allows applications to map large files (including files larger than the actual available amount of system memory) as read-only without hitting <code>ENOMEM</code> errors. (They could do this previously if they specified <code>MAP_NORESERVE</code>, but now it will work even without that flag being specified.) This fixes use of Git on large repositories on systems with limited RAM.</p> <h3 id="build-system">Build system</h3> <p>kallisti5 updated the GCC package built into full Haiku image builds to the current version in the update repositories (13.2).</p> <h3 id="documentation">Documentation</h3> <p>waddlesplash fixed a typo preventing a page on synchronization primitives from appearing properly in the API documentation.</p> <p>PulkoMandy documented the various item-reordering APIs in <code>BMenu</code>.</p> <p>madmax fixed some typos, copy-pasta, and strange grammar in various classes in the API documentation.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>No new changes in-tree this month for ARM or RISC-V, but there was at least some out-of-tree activity.</p> <h3 id="haikuports">HaikuPorts</h3> <p>Begasus updated the R port, and added a port of RKWard, the KDE IDE for R.</p> <p>augiedoggie added an (initially disabled) recipe for <code>fuse-nfs</code>, which (after the userlandfs fixes) seems to work pretty well.</p> <h3 id="are-we-beta5-yet">Are we beta5 yet?</h3> <p>Not yet, no. But I fixed a number of tickets last month which were in the beta5 milestone (and determined some more could be de-prioritized out of the milestone), putting a good dent in the number still to be completed&hellip;</p> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p>RFC Coding Guidelines: Formatting Class Member Declarationshttps://www.haiku-os.org/blog/nielx/2024-02-05_rfc_coding_guidelines_formatting_class_member_declarations/nielxMon, 05 Feb 2024 16:13:58 +0000https://www.haiku-os.org/blog/nielx/2024-02-05_rfc_coding_guidelines_formatting_class_member_declarations/<p>This RFC proposes to change the Haiku <a href="https://www.haiku-os.org/development/coding-guidelines">coding guidelines</a> to change the formatting of variable and member declarations from a <strong>Table Class Member Declaration Style</strong>, to a <strong>Normalized Declaration Style</strong> or a <strong>Aligned Declaration Style</strong>. The arguments are that (1) the current format has severe limitations which limits the aesthetic value of the current formatting, especially when modern C++ language features are used, and (2) it is not a good use of the time of Haiku&rsquo;s contributors to modify and maintain custom logic in the <code>haiku-format</code> tool (derived from <code>clang-format</code>). If the proposal is adopted, any new code contributions will have to use the new formatting style, and contributors are required to reformat any declarations that they modify.</p> <p>Haiku currently does not have a formal RFC process. Please see the end of this document about the proposed procedure.</p> <div class="alert alert-warning"><p><strong>Proposal Withdrawn</strong><br/> The proposal to change the formatting style of function and variable declarations has been withdrawn by the author on 16 March 2024. </p> </div> <h2 id="change-log">Change Log</h2> <p>The <a href="https://github.com/haiku/website/blob/53e092e580ee12c06a3d20ce608e4b55e49c420a/content/blog/nielx/2024-02-05_rfc_coding_guidelines_formatting_class_member_declarations.md">initial version</a> of this article was published on 5 February 2024. After consulting with the community on the forum, the article was updated into the version you are currently reading.</p> <p>Note the following changes:</p> <ul> <li>The proposal now covers all function and variable declarations, so including the ones outside of the class definition. This is both for pragmatic reasons (changing <code>haiku-format</code> would automatically apply to non-class members) as well as the finding that currently this is also not consistent.</li> <li>In course of the community discussion, there was a desire to see if there was an alternative that kept the spirit of the aligned declarations. This proposal therefore gives two options, namely the <strong>Normalized Declaration Style</strong> and the <strong>Aligned Declaration Style</strong>. Both can be implemented with <code>haiku-format</code> as it currently is.</li> <li>The discussion also reviewed the implementation process. My takeaway was that there is a strong preference for a single style (and to not have a legacy option), so the implementation kill the exception for the legacy style. There was no consensus on whether a big one time reformat was in order, so this proposal punts on that point and only applies to new code.</li> </ul> <h2 id="background-coding-guidelines">Background: Coding Guidelines</h2> <p>Like many open source projects and companies, Haiku has <a href="https://www.haiku-os.org/development/coding-guidelines">coding guidelines</a> that standardize the practices around code formatting, naming conventions, and the use of C++ language features. As our coding guidelines state, the main aim is to have the &ldquo;Haiku code base (&hellip;) remain clean, uniform, and easy to read and maintain.&rdquo; If done right, a uniform code style will reduce the cognitive load for contributors when they are modifying the code base, and for core developers while reviewing contributions.</p> <p>In the field of code guidelines, a lot of tooling has been introduced to help automate the process of writing code in the right style. A common tool for code formatting is <a href="https://clang.llvm.org/docs/ClangFormat.html">clang-format</a>, which is part of the LLVM project. Since 2018, owenca has been maintaining <a href="https://github.com/owenca/haiku-format/">haiku-format</a>, which is a patched version of clang-format that both incorporates some haiku-specific formatting changes, as well as codifying the Haiku style by default. In 2021, there were also two GSOC projects. In <a href="https://www.haiku-os.org/blog/saloni/">saloni worked on a project</a> to implement some more of Haiku&rsquo;s style in clang-format. And <a href="https://www.haiku-os.org/blog/ritz/">ritz</a> worked on a <a href="https://www.haiku-os.org/blog/ritz/">bot</a> that would automate running over changes submitted to Gerrit.</p> <p>Recently, I have taken the initiative for the <a href="https://github.com/haiku/haiku-format-bot/">Haiku Format Bot</a>. It leverages <code>haiku-format</code> to check changes submitted to Gerrit. I have taken inspiration from ritz' initiative, though I have concluded that the use of concourse is probably overkill, and it could be implemented with a lighter, stateless, web service that can be ran independently from concourse. Read more about the plan for that tool in the <a href="https://discuss.haiku-os.org/t/introducing-haiku-format-bot-0-1/14485">0.1 announcement</a>.</p> <h2 id="the-issue-with-class-and-struct-member-formatting">The Issue with Class and Struct Member formatting</h2> <p>Currently, <code>haiku-format</code> does not implement our table class member declaration style formatting rules. That is not for the lack of trying: the 2021 GSoC project by saloni has an implementation of <a href="https://github.com/saloniig/llvm-project/commit/2892ce0387f356576b1de2ea07cadeda79ac399b">Haiku&rsquo;s member formatting rules</a>. That code was never adopted by <code>haiku-format</code> though, and it is currently not maintained. Inclusion in <code>haiku-format</code> has been discussed, but has never been completed due to the GSoC project being finished.</p> <p>Could this change be included in <code>haiku-format</code>? Potentially, yes, though not with a bit of work. It is a few years out of sync with the newest changes in <code>clang-format</code>, and the patch also does not include any (automated) tests to validate the logic. Furthermore, there is also a philosophical argument whether this type of parsing fits in the view of how <code>clang-format</code> should work. As owenca notes, the reason that the patch is complex, is because it has to work around the fact that<code>clang-format</code> only uses LLVM&rsquo;s C++ lexer, but not the part where it parses the AST and thus infers meaning of the tokens (<a href="https://github.com/owenca/haiku-format/issues/19#issuecomment-860130695">see this note on Github</a>). This means that for our type of formatting to work, a lot of additional logic to identify whether something is a modifier, a return type or the name of a member is added.</p> <p>This has lead me to the point on whether or not it is worth putting in the effort codifying this part of our coding style into <code>haiku-format</code>. Ultimately, I do not think it is worth the effort to introduce and maintain this change. After careful consideration, I have ergonomic and economic arguments to break with the existing style on this point. I will first review the current rule (and its interpretation). After that I will illustrate with examples why the rules actually might introduce inconsistency. Finally I will discuss why I think it is not economical for a small, volunteer driven project to invest time in complex customization and maintenance of <code>clang-format</code>.</p> <h3 id="review-of-the-table-class-member-style">Review of the Table Class Member Style</h3> <p>Haiku has codified a particular style when it comes to declaring classes or structs. The example below is given in the code guidelines:</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">Foo</span> : <span style="color:#6ab825;font-weight:bold">public</span> Bar { <span style="color:#6ab825;font-weight:bold">public</span>: Foo(int32); <span style="color:#6ab825;font-weight:bold">virtual</span> ~Foo(); <span style="color:#6ab825;font-weight:bold">virtual</span> <span style="color:#6ab825;font-weight:bold">void</span> <span style="color:#447fcf">SomeFunction</span>(SomeType* argument); <span style="color:#999;font-style:italic">// indent long argument lists by a tab: </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* <span style="color:#447fcf">FunctionLotsOfArguments</span>(<span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* name, <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* path, <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* user); <span style="color:#6ab825;font-weight:bold">private</span>: int32 _PrivateMethod(); <span style="color:#6ab825;font-weight:bold">static</span> int32 <span style="color:#447fcf">_SomeStaticPrivateMethod</span>(); <span style="color:#999;font-style:italic">// Redundant private: to separate the fields from the methods: </span><span style="color:#999;font-style:italic"></span><span style="color:#6ab825;font-weight:bold">private</span>: <span style="color:#6ab825;font-weight:bold">volatile</span> int32 fMember; <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* fPointerMember; }; </code></pre></div><p>As you can see, the resulting code is a table-like formatted class definition. I am unsure whether the rule is actually explicitly spelled out somewhere, so I am going to quote <a href="https://github.com/owenca/haiku-format/issues/19#issuecomment-852410195">pulkomandy&rsquo;s definition</a> which I use as a guideline myself:</p> <blockquote> <p>The basic idea is:</p> <p>indented 1 tab: qualifiers (volatile, virtual, static)</p> <p>indented 3 tabs: type of fields and return type of functions</p> <p>indented 8 tabs: function or field name</p> </blockquote> <p>And indeed, if you look at any of our <a href="https://git.haiku-os.org/haiku/tree/headers/os?h=r1beta4">public API headers</a>, you will see that this generally works out to give decent table-like class definitions. However, a closer look at the headers will also reveal the rule is a bit more flexible than one may think. As PulkoMandy in the same comment clarified:</p> <blockquote> <p>In some cases we move the function and field names to a further tab stop (9 or 10) to keep things aligned even when there are long return types. Sometimes we move them back to the left when return types are short, but function or field names are very long.</p> </blockquote> <p>Whether that is the rule or not is not entirely clear; the coding guidelines are not explicit about this, they just give the example of the main rule as illustrated by the 1 tab, 3 tabs, 8 tabs guideline. For the purpose of this RFC, that explicit definition is taken as the starting point.</p> <h3 id="why-the-current-rule-introduces-inconsistencies">Why the Current Rule Introduces Inconsistencies</h3> <p>The current rule introduces a table-like format for class members. In the (quick) historical search, I did not find the exact origins of this style. From my perspective, it works particularly well under the following conditions:</p> <ul> <li> <p>It is used in simple, non-nested classes and structs with (mostly) member functions and member data.</p> </li> <li> <p>It works best when the class merely declares its members, but does not define them. This works best when there is a clear separation between a definition of a class with the declaration of its members in the header file, and a definition of the members in the source file.</p> </li> <li> <p>It works well with standard Haiku&rsquo;s C++style, which (generally) does not use nested classes or exceptions. There is also very limited use of templated types.</p> </li> </ul> <p>Admittedly, when a class or a struct can be implemented in such a way that it does not break any of the rules, the resulting class definition looks really quite nice and organized. However, not all classes are neat. And I would argue, that it is likely that we are going to introduce more complex C++ constructs into the codebase, in which case the style starts to break down and introduces inconsistencies.</p> <h4 id="column-length-violations">Column Length Violations</h4> <p>The first group of inconsistencies are when the code leads to column length violations. The first type of violation is when the qualifier is oversized. According to the rule, tabs 2 and 3 are for leading qualifiers and specifiers. In order for content to fit in nicely, this means 2*4-1 = 7 characters. The following examples show how this breaks:</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">LeadingQualifierSpecifierExample</span> { <span style="color:#999;font-style:italic">// &#39;good&#39; example </span><span style="color:#999;font-style:italic"></span> int32 fData1; <span style="color:#999;font-style:italic">// volatile makes the return type unaligned. </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">volatile</span> int32 fData2; <span style="color:#999;font-style:italic">// so does [[nodiscard]] </span><span style="color:#999;font-style:italic"></span> <span style="color:#bbb">[[nodiscard]]</span> int32 fData3; }; </code></pre></div><p>Tabs 4 through 7 are for the return value in case of a member function, or a type in case of member data. This means 5*4-1 = 19 characters. If the return value is a <code>const</code>, that reduces the space down to 13 characters. In the worst case, the return type is an r-value, so the appended <code>&amp;&amp;</code> reduces the space to 11 characters. This limitation currently is not too bad in existing Haiku code, but I expect that future APIs may make more use of language features like <code>std::expected&lt;&gt;</code> (15 characters, and then it still needs two type parameters) or <code>std::shared_ptr&lt;&gt;</code> (17 characters without the type).</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">LongReturnTypeExample</span> { <span style="color:#6ab825;font-weight:bold">public</span>: <span style="color:#999;font-style:italic">// uncomplicated return value </span><span style="color:#999;font-style:italic"></span> BKey Key(); <span style="color:#999;font-style:italic">// existing problematic return type </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">const</span> BMessageQueue* <span style="color:#447fcf">Key</span>() <span style="color:#999;font-style:italic">// potential future complex return type </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">const</span> std::expected&lt;BKey, BError&gt; Key(); }; </code></pre></div><h4 id="no-focus-on-anything-after-the-member-name">No Focus on Anything After the Member Name</h4> <p>The second group of issues with the current style is that it emphasizes the qualifiers and specifiers at the beginning of the declaration, but it de-emphasizes the parts after the parameter list. See the following list for a collection of important specifiers and qualifiers, and see how visually they are second order of importance.</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">MoreAtTheEndExample</span> { <span style="color:#6ab825;font-weight:bold">public</span>: <span style="color:#999;font-style:italic">// emphasis in this line is on the return type and the name of the member function </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> BKey Key(<span style="color:#6ab825;font-weight:bold">const</span> BString&amp; key, <span style="color:#6ab825;font-weight:bold">bool</span> searchWell, int32 offset); <span style="color:#999;font-style:italic">// however, C++98 already puts some important information at the end... </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> BKey <span style="color:#447fcf">Key</span>(<span style="color:#6ab825;font-weight:bold">const</span> BString&amp; key, <span style="color:#6ab825;font-weight:bold">bool</span> searchWell, int32 offset) <span style="color:#6ab825;font-weight:bold">const</span> =<span style="color:#3677a9">0</span>; <span style="color:#999;font-style:italic">// and modern C++ adds much more </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> BKey <span style="color:#447fcf">Key</span>(<span style="color:#6ab825;font-weight:bold">const</span> BString&amp; key, <span style="color:#6ab825;font-weight:bold">bool</span> searchWell, int32 offset) <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">noexcept</span> <span style="color:#6ab825;font-weight:bold">override</span>; <span style="color:#6ab825;font-weight:bold">private</span>: <span style="color:#999;font-style:italic">// and what about initializers for data members? </span><span style="color:#999;font-style:italic"></span> BKeyChain* fKeyChain = <span style="color:#6ab825;font-weight:bold">nullptr</span>; int32 fCount = -<span style="color:#3677a9">1</span>; }; </code></pre></div><h4 id="unhandled-language-constructs">Unhandled Language Constructs</h4> <p>The third and final group of issues are that the language allows for several language features, that do not quite align with the rules on member function and member data column layout. Think of the following:</p> <ul> <li> <p>Friend classes, friend functions, typedefs, and aliases.</p> </li> <li> <p>Subclass declarations within the parent class.</p> </li> <li> <p>Member value initialization</p> </li> </ul> <p>Furthermore, while the current style also does not seem to allow for the definition of member functions within a class definition (<em>with the exception of empty methods</em>), if the rules around member definitions inside the class change, they will also mesh less well with the table member declaration formatting.</p> <h4 id="summary-about-ergonomic-limitations-of-the-style">Summary about Ergonomic Limitations of the Style</h4> <p>Above, I have tried to show how the current formatting rules does not cleanly deal with column length violations (and how that may be more frequent in the future), how it de-emphasizes important parts of member definitions, and how it does not account for various language features that we may want to use in our code. Note that I am aware many of these problems can be &lsquo;solved&rsquo; or worked around, I have done so in my own Haiku contributions. But the main point is that as our code will eventually adopt more modern C++ concepts, it is likely that the advantage of the column layout is diminished.</p> <h3 id="why-keeping-the-custom-format-is-not-economical">Why Keeping the Custom Format is not Economical</h3> <p>The second part of my argument on why to ditch the existing custom class member formatting, is of an economical nature. Technically, it is not impossible to teach <code>clang-format</code> to apply our rules, as demonstrated by the GSoC 2021 patch that implements <a href="https://github.com/saloniig/llvm-project/commit/2892ce0387f356576b1de2ea07cadeda79ac399b">Haiku&rsquo;s member formatting rules</a>. As discussed in the previous paragraph though, this patch needs to be brought up to date, needs to have automated test cases added, and ultimately needs to be maintained in the future. I would argue, though, that this is not worth the effort.</p> <p>In order to make this case, I start with the following assumptions:</p> <ol> <li> <p>Haiku is a volunteer-driven open source project, where individual contributors put in their time in exchange for value they derive from their contributions. While the value is personal to everyone, I suspect a common shared value is the pride of contributing to an experimental alternative desktop operating system.</p> </li> <li> <p>As an Open Source project, Haiku shares the values of community, and collective improvement through the competition of ideas to generally improve the overall software world. OSS projects absorb, assimilate and depend on the best ideas in the ecosystem, and contribute in parts where they think they can improve the ecosystem. Haiku&rsquo;s contributions are in the space of OS Design, UI Design and Application Design.</p> </li> <li> <p>A common code style is very important in an OSS project, and in principle should be enforced.</p> </li> </ol> <p>Given the three assumptions, I would say that the value of Haiku contributors is not in creating and maintaining extended changes to <code>clang-format</code> in order to satisfy a particular historical code style format. I personally do not have any interest in digging into the internals of <code>clang-format</code> in order to implement and maintain a solution. Given the fact that two and a half years after GSOC 2021 no one else has, indicates that this is not where our volunteer developers think our project&rsquo;s contributions add value.</p> <p>Furthermore, if you accept point 3 about the importance of code style, it must then also be true that our developers spending a lot of time checking code formatting, is not the best use of their time. Getting <code>haiku-format</code> off the shelf and putting it in use on a bigger scale, unlocks the value of the tool, and the value of the ideas created during GSoC 2021.</p> <p>It is for those reasons that I consider us spending time adding support for our particular formatting of class member declarations, is uneconomic: it is not where individual contributors to Haiku will get their pleasure, nor will it be where Haiku delivers value to the larger community.</p> <h2 id="two-proposals-for-a-new-style">Two proposals for a new style</h2> <p>This section introduces two options for a new style, both of which can be configured with the current version of <code>haiku-format</code>. The <strong>Normalized Declaration Style</strong> is actually the current style of <code>haiku-format</code>. It does away with any form of table formatting, and is the simplest form when writing code. The second style is the <strong>Aligned Declaration Style</strong>, which retains some form of vertical alignment to enhance reading code, though it will require developers to do (re-)alignment of code when they make changes. This, of course, can be done with <code>haiku-format</code>.</p> <p>Note that both styles will apply beyond member declarations of classes and structs, as they will apply to any function or variable declaration anywhere in the source code!</p> <h3 id="option-1-normalized-declaration-style">Option 1: Normalized Declaration Style</h3> <p>As part of this proposal, I will demonstrate <code>haiku-format</code>&rsquo;s current formatting for function and variable declarations. I will call this style the <strong>Normalized Declaration Style</strong>. The example shows the existing formatting of a simplified <code>BHandler</code> declaration and some function declarations:</p> <p>After applying <code>haiku-format</code>:</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">BHandler</span> : <span style="color:#6ab825;font-weight:bold">public</span> BArchivable { <span style="color:#6ab825;font-weight:bold">public</span>: BHandler(<span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* name = <span style="color:#24909d">NULL</span>); <span style="color:#6ab825;font-weight:bold">virtual</span> ~BHandler(); <span style="color:#999;font-style:italic">// BHandler guts. </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> <span style="color:#6ab825;font-weight:bold">void</span> <span style="color:#447fcf">MessageReceived</span>(BMessage* message); BLooper* <span style="color:#447fcf">Looper</span>() <span style="color:#6ab825;font-weight:bold">const</span>; <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* <span style="color:#447fcf">Name</span>() <span style="color:#6ab825;font-weight:bold">const</span>; <span style="color:#999;font-style:italic">// Long declaration, split over multiple lines </span><span style="color:#999;font-style:italic"></span> status_t <span style="color:#447fcf">Launch</span>(<span style="color:#6ab825;font-weight:bold">const</span> entry_ref* ref, <span style="color:#6ab825;font-weight:bold">const</span> BMessage* initialMessage = <span style="color:#24909d">NULL</span>, team_id* _appTeam = <span style="color:#24909d">NULL</span>) <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">volatile</span> <span style="color:#6ab825;font-weight:bold">noexcept</span>; <span style="color:#6ab825;font-weight:bold">private</span>: <span style="color:#6ab825;font-weight:bold">typedef</span> BArchivable _inherited; <span style="color:#6ab825;font-weight:bold">friend</span> <span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">BLooper</span>; <span style="color:#6ab825;font-weight:bold">friend</span> <span style="color:#6ab825;font-weight:bold">inline</span> int32 <span style="color:#447fcf">_get_object_token_</span>(<span style="color:#6ab825;font-weight:bold">const</span> BHandler*); std::list&lt;int32&gt; fTokens; <span style="color:#6ab825;font-weight:bold">char</span>* fName; }; <span style="color:#6ab825;font-weight:bold">extern</span> DIR* <span style="color:#447fcf">fs_open_index_dir</span>(dev_t device); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">int</span> <span style="color:#447fcf">fs_close_index_dir</span>(DIR* indexDirectory); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">struct</span> <span style="color:#447fcf;text-decoration:underline">dirent</span>* <span style="color:#447fcf">fs_read_index_dir</span>(DIR* indexDirectory); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">void</span> <span style="color:#447fcf">fs_rewind_index_dir</span>(DIR* indexDirectory); BLooper* BLooper::Looper() <span style="color:#6ab825;font-weight:bold">const</span> { <span style="color:#999;font-style:italic">// example of the existing style of declarations inside functions </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">int</span> count; <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* name; int32 index = <span style="color:#3677a9">0</span>; } </code></pre></div><p>The example above is generated by <code>haiku-format</code> as it is right now: it does not require any additional configuration or changes. The coding guidelines would be amended as follows.</p> <p><strong>Declarations</strong></p> <ul> <li><em>Declarations</em> of types, variables, functions, methods, aliases, friends, etcetera are on a single line, with each token in the declaration separated spaces according to the rules in the &lsquo;Indenting and Whitespace&rsquo; section.</li> </ul> <h3 id="option-2-aligned-declaration-style">Option 2: Aligned Declaration Style</h3> <p>Additionally, <code>haiku-format</code> can be configured to support an alternative style. I will call this style the <strong>Aligned Declaration Style</strong>. The example shows the existing formatting of a simplified <code>BHandler</code> declaration and some function declarations:</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c++" data-lang="c++"><span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">BHandler</span> : <span style="color:#6ab825;font-weight:bold">public</span> BArchivable { <span style="color:#6ab825;font-weight:bold">public</span>: BHandler(<span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* name = <span style="color:#24909d">NULL</span>); <span style="color:#6ab825;font-weight:bold">virtual</span> ~BHandler(); <span style="color:#999;font-style:italic">// BHandler guts. </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">virtual</span> <span style="color:#6ab825;font-weight:bold">void</span> <span style="color:#447fcf">MessageReceived</span>(BMessage* message); BLooper* <span style="color:#447fcf">Looper</span>() <span style="color:#6ab825;font-weight:bold">const</span>; <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* <span style="color:#447fcf">Name</span>() <span style="color:#6ab825;font-weight:bold">const</span>; <span style="color:#999;font-style:italic">// Long declaration, split over multiple lines </span><span style="color:#999;font-style:italic"></span> status_t <span style="color:#447fcf">Launch</span>(<span style="color:#6ab825;font-weight:bold">const</span> entry_ref* ref, <span style="color:#6ab825;font-weight:bold">const</span> BMessage* initialMessage = <span style="color:#24909d">NULL</span>, team_id* _appTeam = <span style="color:#24909d">NULL</span>) <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">volatile</span> <span style="color:#6ab825;font-weight:bold">noexcept</span>; <span style="color:#6ab825;font-weight:bold">private</span>: <span style="color:#6ab825;font-weight:bold">typedef</span> BArchivable _inherited; <span style="color:#6ab825;font-weight:bold">friend</span> <span style="color:#6ab825;font-weight:bold">class</span> <span style="color:#447fcf;text-decoration:underline">BLooper</span>; <span style="color:#6ab825;font-weight:bold">friend</span> <span style="color:#6ab825;font-weight:bold">inline</span> int32 <span style="color:#447fcf">_get_object_token_</span>(<span style="color:#6ab825;font-weight:bold">const</span> BHandler*); std::list&lt;int32&gt; fTokens; <span style="color:#6ab825;font-weight:bold">char</span>* fName; }; <span style="color:#6ab825;font-weight:bold">extern</span> DIR* <span style="color:#447fcf">fs_open_index_dir</span>(dev_t device); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">int</span> <span style="color:#447fcf">fs_close_index_dir</span>(DIR* indexDirectory); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">struct</span> <span style="color:#447fcf;text-decoration:underline">dirent</span>* <span style="color:#447fcf">fs_read_index_dir</span>(DIR* indexDirectory); <span style="color:#6ab825;font-weight:bold">extern</span> <span style="color:#6ab825;font-weight:bold">void</span> <span style="color:#447fcf">fs_rewind_index_dir</span>(DIR* indexDirectory); BLooper* BLooper::Looper() <span style="color:#6ab825;font-weight:bold">const</span> { <span style="color:#999;font-style:italic">// aligned declarations inside functions as well! </span><span style="color:#999;font-style:italic"></span> <span style="color:#6ab825;font-weight:bold">int</span> count; <span style="color:#6ab825;font-weight:bold">const</span> <span style="color:#6ab825;font-weight:bold">char</span>* name; int32 index = <span style="color:#3677a9">0</span>; } </code></pre></div><p>The example above is generated by <code>haiku-format</code> as it is right now: it does not require any additional configuration or changes. The coding guidelines would be amended as follows.</p> <p><strong>Indention and whitespace</strong> (amendment)</p> <ul> <li>Use spaces to align tokens within a line (such as aligned function declarations).</li> </ul> <p><strong>Declarations</strong></p> <ul> <li><em>Declarations</em> of variables and functions are on a single line.</li> <li>When two or more declarations are consequitive they form a block. Within a block, the identifiers of the variable or function are vertically aligned. The point of alignment is based on the identifier that is positioned at the rightmost position in the line.</li> <li>Constructors and destructors are not aligned (even when the destructor is <code>virtual</code>).</li> <li>A block ends when there is a comment, or a statement that is not a variable or function declaration. In a class, a block is also ends when an access modifier (public, protected, private) is found. Empty lines do <em>not</em> break a block, so that they can be used to further group declarations within a block.</li> <li>Other types of declarations (such as types and aliases) are on a single line, with each token in the declaration separated spaces according to the rules in the &lsquo;Indenting and Whitespace&rsquo; section.</li> </ul> <h2 id="proposal--implementation-of-the-new-rules">Proposal &amp; Implementation of the New Rules</h2> <p>Given the arguments above, I propose changing the rule to use one of the two styles described above, which both are currently implemented in the <code>clang-format</code> (and by extension <code>haiku-format</code>) rules.</p> <p>When adopting either of these proposals, the existing formatting becomes obsolete. In order not to force a rewrite of the entire code base, the rule only applies to new contributions.</p> <p>Therefore, <strong>any new code must use the formatting of class and struct members as <code>haiku-format</code> and the (updated) Haiku code style guidelines</strong>.</p> <p>The implementation of this proposal does <em>not</em> require going through the entire code base at this time and reformatting all declarations, however if contributors are introducing changes to the code, they will share the responsibility to reformat any existing declarations. This means:</p> <ul> <li>If a contributor makes a change that touches on a struct or class declaration in a source code file or in a header file that is not shared, then reformatting that struct or class is a requirement of the change being accepted. Likewise, if their contribution changes a declaration outside a class that is part of a block of declarations, the entire block must be reformatted.</li> <li>This may mean that in some cases, it might be beneficial for a contributor to break up their change into two commits, a reformat commit and a commit with the actual changes, to help code review.</li> <li>Likewise, if a contributor is making a series of changes in a particular module in the code, it might be more efficient if they introduce an overall reformatting change touching all files for that module, in order to make reviews of subsequent changes easier.</li> </ul> <h2 id="out-of-scope">Out of Scope</h2> <p>This RFC is limited to changing the formatting guidelines for member declarations within class definitions.</p> <p>Out of scope for this RFC are:</p> <ul> <li> <p>The discussion of whether any of the language features used as examples above must be added to the style guide.</p> </li> <li> <p>Doing a one-time reformat of (parts) of the codebase.</p> </li> <li> <p>Any proposal about the further use of the Haiku Format Bot.</p> </li> </ul> <h2 id="next-steps">Next Steps</h2> <p>Haiku currently does not have a formal RFC process. This proposal will follow the following process:</p> <ul> <li> <p>This proposal is published as a blog post on the author&rsquo;s personal blog on <a href="http://www.haiku-os.org">www.haiku-os.org</a>.</p> </li> <li> <p>When published, the haiku-development mailing list will be notified of the proposal.</p> </li> <li> <p>The proposal can be discussed on the Haiku Discussion forum. The Haiku website will link the comment thread to the RFC.</p> </li> <li> <p>The discussion will be open for comments for at least 7 days after the mailing list is notified.</p> </li> <li> <p>When the author thinks the discussion is concluded, they will inform the haiku-development mailing list of the outcome. Outcomes can be (a) consensus reached, (b) vote required or (c) proposal withdrawn. In the case of outcome (a), core contributors can request a vote. In case of outcome (c), another contributor can adopt the proposal and continue discussion or put it up for a vote.</p> </li> </ul>Haiku Activity & Contract Report, December 2023https://www.haiku-os.org/blog/waddlesplash/2024-01-12_haiku_activity_contract_report_december_2023/waddlesplashFri, 12 Jan 2024 22:00:00 -0400https://www.haiku-os.org/blog/waddlesplash/2024-01-12_haiku_activity_contract_report_december_2023/<p>This report covers hrev57429 through hrev57493.</p> <h3 id="tcp-fixes">TCP fixes</h3> <p>The most notable series of fixes last month were in the TCP implementation. There have been problems and inefficiencies known in it for some time, but following reports that they could be reproduced even on loopback (not just on LAN), I decided to spend time investigating.</p> <p>First, I revived the <code>tcp_shell</code> (which was last used in 2008 or thereabouts.) This is a test harness for the TCP implementation that allows it to be run entirely in userland. (At present, it can&rsquo;t send or receive data outside itself, so it&rsquo;s mostly useful for testing the TCP implementation against itself.) This is highly useful, as it allows running the TCP implementation with a real debugger attached (and of course, edit-compile-run cycles are much shorter than when a base system package must be built and installed to test it.) After I&rsquo;d gotten it working again, I added support for writing <code>.pcap</code> files to it, allowing the results of such userland tests to be analyzed in Wireshark (<code>tcpdump</code> can be used to generate <code>pcap</code> files for the real loopback interface), and a command to send large amounts of data across the internal connection.</p> <p>After that, I set in on understanding just what was happening to cause connections speeds to degrade so severely (and with little CPU being used.) I eventually discovered that lots of duplicate ACKs were being sent when they shouldn&rsquo;t. These are usually used as indicators of congestion, lost packets, or other problems, and so they caused the sending side of the connection to increasingly slow its transmissions. Fixing this problem solved some of the performance degradation.</p> <p>The other issue turned out to be that when the send window size dropped to 0, occasionally the sending side would not immediately resume transmission when it received a window update, but would wait for the &ldquo;persist&rdquo; timer to fire. This turned out to be a long-standing bug for which a fix had been attempted years before, but it had missed a crucial case which resulted in it being mostly ineffective.</p> <p>These two problems resolved (and some other minor refactors and cleanups done along the way), TCP throughput on loopback (on a single-core VM) increased from an unsteady ~45Mbit/sec to a solid 5.4 Gbit/sec. (On multi-core machines the difference will be much less dramatic; it was already in the Gbit/sec range there.) Users have reported improvements in real-world traffic speeds, too, but at least here there&rsquo;s still much more work to be done: the whole implementation could use a refactor, and then some work to take better advantage of TCP features like SACK, congestion-control, and window-scale. I intend to work on some of these, so stay tuned for that.</p> <h3 id="applications">Applications</h3> <p>jscipione removed a stray separator from a context menu in Tracker&rsquo;s FilePanel (the system-wide open/save dialogs), made packagefs volumes display as size &ldquo;-&rdquo; instead of an incorrect (small) size, and performed some code cleanup. He also made changes to allow &ldquo;Get info&rdquo; being invoked from the File menu with no selection (in this case it opens the info window for the current directory), and fixed the logic checking defaults for thumbnail support in Tracker settings. Later, he moved all the default Tracker settings constants into one location (they previously had been duplicated in various parts of the source code.)</p> <p>OscarL fixed RAM size not updating properly in the replicant version of AboutSystem. Later, jscipione reworked AboutSystem to not archive views in the replicant at all, but instead to recreate them freshly every time the replicant is instantiated, and made modifications so that the extra border is hidden when &ldquo;Show replicants&rdquo; is turned off.</p> <p>jscipione adjusted some strings for consistency in Keymap, and fixed some regressions in the Modifier Keys window from when Caps Lock was added as a modifier.</p> <p>humdinger fixed the bookmark bar not displaying in WebPositive under non-English locales.</p> <p>waddlesplash fixed TUN/TAP interfaces appearing with the &ldquo;Wi-Fi&rdquo; device icon in the Network preflet.</p> <p>mt fixed a file descriptor leak in Expander.</p> <p>humdinger replaced the &ldquo;window&rdquo; icons used in Deskbar. (The new icons have much clearer indicators for windows on other workspaces.)</p> <h3 id="command-line-tools">Command line tools</h3> <p>kallisti5 fixed an incorrect format string in <code>remote_disk_server</code>.</p> <p>waddlesplash adjusted <code>pkgman</code> to use &ldquo;natural&rdquo; sorting in <code>pkgman search</code> output (e.g. so that &ldquo;llvm12&rdquo; will appear after &ldquo;llvm9&rdquo;.)</p> <h3 id="kits">Kits</h3> <p>jscipione modified BListView to update the selected item on mouse-up, <em>if</em> it was different than the item selected on mouse-down. This is useful e.g. when holding the mouse while scrolling through a list of items. He then implemented auto-scroll on &ldquo;drag&rdquo;, and made a number of fixes to various applications to take advantage of the new scrolling logic: DataTranslators, Icon-O-Matic, etc.</p> <p>jscipione added logic to BMenuItem to support visually representing &ldquo;backspace&rdquo; and &ldquo;delete&rdquo; shortcuts in menus.</p> <p>jscipione cleaned up some code in BTextView, and then added logic to prevent scrolling if all text was already visible in the view.</p> <p>PulkoMandy implemented some more features in HaikuDepot&rsquo;s custom TextView (which is a prototype replacement for BTextView with a much cleaner design), including clickable text (for hyperlinks), underlines, and forcing a relayout of the document when things change.</p> <p>Freaxed fixed resizing glitches in BColumnListView for especially wide columns.</p> <p>X512 fixed attempts to bind <code>AF_LOCAL</code> sockets in <code>BAbstractSocket</code> when connecting.</p> <p>korli fixed a memory management regression in the FFmpeg media add-on that was leading to crashes when playing media.</p> <p>PulkoMandy refactored some parts of the FFmpeg media add-on to remove usage of deprecated functions and prepare the way for using newer versions of FFmpeg.</p> <h3 id="servers">Servers</h3> <p>X512 modified app_server to clear view backgrounds immediately on expose. This will reduce redraw artifacts when applications are responding slowly (or are entirely hung.)</p> <h3 id="drivers">Drivers</h3> <p>korli fixed some problems with the global-lock code in our ACPICA glue code, and cleaned up some of its build rules (as well as the build rules for some other kernel add-ons, at the same time.) PulkoMandy fixed a tracing problem in the glue code, and then updated to ACPICA version 20230628. korli then adjusted the ACPI module to export some more attributes into the device manager.</p> <p>nephele attempted to fix a problem with brightness-control handoff in the <code>radeon_hd</code> driver.</p> <p>korli fixed incorrect keys being reported on select (USB) HID keyboards with &ldquo;minimum&rdquo; values specified in their reports.</p> <p>X512 (with some modifications from waddlesplash) adjusted the <code>poke</code> driver to map physical memory into the team&rsquo;s address space rather than the kernel&rsquo;s, meaning it will be automatically cleaned up on team exit.</p> <p>mt fixed a use-after-free in an error branch in the virtio_block intialization code.</p> <p>waddlesplash refactored network device statistics to be updated (mostly) within the stack itself, as opposed to the various device modules (ethernet, etc.) Now the only case where a device module has to update the device statistics directly is when packets are dropped in receive routines. This also fixes another case where loopback statistics were not being updated properly.</p> <h3 id="file-systems">File systems</h3> <p>mt fixed a duplicate expression in the FAT driver.</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>mmu_man renamed <code>IFT_TUN</code> to <code>IFT_TUNNEL</code> (to match the name of the constant on the BSDs.)</p> <p>waddlesplash made the <code>B_CLONEABLE_AREA</code> protection flag modifiable from userland (through <code>set_area_protection</code>), and cleaned up some bits of the kernel implementation of area-protection modification.</p> <p>PulkoMandy made <code>regex.h</code> use <code>_DEFAULT_SOURCE</code> to protect its non-standard features (instead of a non-standard GNU preprocessor define.)</p> <h3 id="build-system">Build system</h3> <p>PulkoMandy fixed the build of the launch_daemon test applications.</p> <h3 id="documentation">Documentation</h3> <p>kallisti5 added a documentation page detailing how to get code completion in IDEs for the Haiku source tree.</p> <p>PulkoMandy added a page describing the source tree&rsquo;s layout (mostly copied from the website, and the old page obsoleted.) He also wrote a page documenting kernel synchronization primitives for the Haiku Book.</p> <p>PulkoMandy added a Python script that generates <code>dot</code> format graphs for dependencies from a set of HPKG files.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>davidkaroly fixed initial page mapping in the EFI loader for RISC-V.</p> <h3 id="haikuports">HaikuPorts</h3> <p>A new version of Genio (the native Haiku IDE) is now available. There are also a lot of new KDE packages (including RKWard, an IDE for the R language.)</p> <h3 id="thats-all-folks">That&rsquo;s all, folks!</h3> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p>Haiku Activity & Contract Report, November 2023 (ft. VPN support)https://www.haiku-os.org/blog/waddlesplash/2023-12-12_haiku_activity_contract_report_november_2023/waddlesplashTue, 12 Dec 2023 22:30:00 -0400https://www.haiku-os.org/blog/waddlesplash/2023-12-12_haiku_activity_contract_report_november_2023/<p>This report covers hrev57364 through hrev57428.</p> <h3 id="tun-support-for-vpns">TUN support, for VPNs</h3> <p>Last month&rsquo;s highlight is probably the addition of a TUN/TAP driver, which is now included on nightly builds. TUN mode definitely works, while TAP mode has not been tested as much.</p> <p>The TUN/TAP driver was initially written by Swangeon as part of GSoC 2023. It wasn&rsquo;t in a fully working state, however, and had a number of issues still remaining when it was initially merged. waddlesplash picked up the work and performed multiple refactors on it, merging a multiple-driver setup into a single network module that also publishes character devices in devfs, overhauling the buffer queuing logic, fixing and completing the TUN portions of the driver (and making necessary modifications to other parts of the network stack), and performing general cleanups.</p> <p>The driver has been tested with OpenVPN, but other TUN-based VPNs should work also. Some manual configuration is required (as OpenVPN does not yet know how to manage network routes on Haiku.) Here&rsquo;s a template set of commands for setting it up:</p> <div class="highlight"><pre tabindex="0" style="color:#d0d0d0;background-color:#202020;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh"><span style="color:#999;font-style:italic">## starting a VPN:</span> <span style="color:#999;font-style:italic"># create interface</span> ifconfig tun/0 up <span style="color:#999;font-style:italic"># start VPN and connect (config file should mention tun/0)</span> openvpn --config &lt;CONFIG FILE&gt; <span style="color:#999;font-style:italic">## then:</span> <span style="color:#999;font-style:italic"># add static route for VPN traffic (needs to go over existing connection)</span> route add /dev/net/&lt;NIC&gt;/0 &lt;VPN SERVER IP&gt; gw &lt;CURRENT GATEWAY IP&gt; <span style="color:#999;font-style:italic"># delete the existing default gateway</span> route del /dev/net/&lt;NIC&gt;/0 default gw &lt;CURRENT GATEWAY IP&gt; <span style="color:#999;font-style:italic"># add the VPN gateway as default; all traffic should now go over VPN</span> route add tun/0 inet default gw &lt;VPN GATEWAY IP&gt; </code></pre></div><h3 id="applications">Applications</h3> <p>axeld refactored DiskProbe&rsquo;s Find window to use layouts.</p> <p>humdinger added an &ldquo;Open with&hellip;&rdquo; menu to ShowImage, allowing one to open the current image in another supported application without having to switch to Tracker. In addition, he fixed saving images (which was broken on 64-bit.) He also fixed an off-by-one error in image placement, and rearranged some menus for consistency.</p> <p>humdinger fixed Tracker to display half-star ratings on files instead of rounding to the nearest whole star. He also added &ldquo;Reset rating&rdquo; menu actions to MediaPlayer and ShowImage, to allow file ratings to be easily reset.</p> <p>apl made some performance improvements to HaikuDepot&rsquo;s loading procedure, including to defer loading sizes or skip it all together when unnecessary, using buffered reads and reusing text buffers during JSON parsing, and more. He also started a refactor of the data-model generation code, writing a new parser generation script and adjusting the generated code for performance, among other changes.</p> <p>humdinger fixed Mail&rsquo;s &ldquo;Save address&rdquo; command to automatically open &ldquo;People&rdquo; with the email and name fields already populated if no contact file yet exists for that email address. (This involved adjusting &ldquo;People&rdquo; to accept such initial values.)</p> <p>davidkaroly introduced target-endianness handling into Debugger&rsquo;s debug-info handling, potentially allowing debug information from architectures with endiannesses differing from the host&rsquo;s to still be loaded and analyzed. He also introduced routines for reading integers in DWARF of arbitrary size (as some are e.g. 3 bytes, not just 1, 2, 4, or 8.) He also implemented parsing of DWARF5&rsquo;s indirect string and address forms.</p> <p>humdinger reworked OverlayImage to use BAboutWindow, fixing some layout problems.</p> <p>jscipione made tooltips be set on truncated window items in Deskbar, so that the full window title can still be seen if desired.</p> <p>jscipione fixed the enabled/disabled status of right-click actions in Tracker in queries (where results will be in different folders with differing permissions.) He also performed some refactoring, combining some duplicate checks, fixed some other regressions around read/write mode and crashes relating to editing filenames, and made the column titles in queries consistent with the rest of the interface.</p> <p>jscipione fixed keyboard tracking on pressing the &ldquo;Up&rdquo; arrow key in menus. (This means items selected upon pressing the &ldquo;up&rdquo; arrow key can now be activated with the keyboard, not merely ones selected upon pressing the &ldquo;Down&rdquo; arrow key.)</p> <p>waddlesplash fixed saving of thumbnail creation times in filesystem attributes in Tracker, fixing their display in the &ldquo;Get info&rdquo; window as well as resolving some other bugs around too frequent or too infrequent regeneration of thumbnails.</p> <p>jscipione removed the display of &ldquo;Inaccessible memory&rdquo; from AboutSystem. This just wound up confusing too many users as similar metrics aren&rsquo;t displayed on other OSes. (The number can still be seen in the relevant command line tools.) He also consolidated some of the information displays at the same time.</p> <h3 id="command-line-tools">Command line tools</h3> <p>mmu_man added help text to the <code>safemode</code> command line tool (used to detect the value of kernel safemode options.)</p> <h3 id="kits">Kits</h3> <p>korli fixed a bad memory leak in the FFmpeg media plugin that was leaking memory when decoding audio frames.</p> <p>PulkoMandy fixed merging of text spans in HaikuDepot&rsquo;s TextView class (a replacement for the older <code>BTextView</code> in the Interface Kit.)</p> <p>jscipione made the text box values be right-aligned in <code>BColorControl</code>.</p> <p>X512 fixed a swapped-parameters problem in <code>BSocket::Bind</code>.</p> <h3 id="drivers">Drivers</h3> <p>X512 fixed setting area names for <code>/dev/poke</code>&rsquo;s <code>POKE_MAP_MEMORY</code> command, fixing a regression introduced back when the driver was fixed for SMAP.</p> <p>waddlesplash (in a series of commits scattered across the month) refactored the handling of the <code>flags</code> parameter of <code>send</code>/<code>sendmsg</code> and <code>recv</code>/<code>recvmsg</code> inside the network stack. Initially, this was done to fix test failures and hangs in some programs using UNIX domain sockets which passed flags that altered behavior (like <code>MSG_PEEK</code>) but which were not actually handled by the UNIX domain sockets implementation; the initial change just made <code>EOPNOTSUPP</code> be returned when unhandled flags were specified. This, however, wound up uncovering that some applications (including <code>curl</code>) had been passing some (relatively unimportant) unsupported flags, leading to some of those being first implemented in the UNIX domain sockets module, but eventually led to refactoring in the main network sockets module and a variety of flags logic being consolidated there rather than implemented uniquely by every socket module (TCP, UDP, etc.) A number of minor bugs in the implementation of such flags were corrected along the way, along with various code cleanups done to the socket module and others (old FIFO templates code, etc.)</p> <p>PulkoMandy reduced the timeout for PS/2 mouse resets. As the keyboard is initialized after the mouse, this can (in some cases) speed up keyboard initialization.</p> <p>waddlesplash fixed updating of network statistics for both ethernet and loopback drivers. On ethernet, it was not thread-safe, and on loopback, statistics were not updated at all (so now the loopback interface will actually display something other than zero for sent bytes and packets.)</p> <p>waddlesplash fixed compiling some private headers in the FreeBSD compatibility layer in C89 mode. (While the kernel is always built with the newer GCC and thus C11 mode, some applications like <code>wpa_supplicant</code> include these headers and may still be compiled in C89 mode.)</p> <h3 id="file-systems">File systems</h3> <p>korli implemented the necessary hook in the ext2/3/4 driver for <code>fcntl(F_SETFL)</code> to work there.</p> <p>waddlesplash adjusted the ext2/3/4 driver to display a name besides merely &ldquo;ext2&rdquo; (which is confusing as the driver supports ext3 and ext4 features.) PulkoMandy later updated the name again. It will now be displayed as &ldquo;ext4&rdquo; or &ldquo;Linux Extended File System 2/3/4&rdquo; (depending on whether the short or the long name is used.)</p> <p>kallisti5 added some default initializers and added a missing check in BFS, following on suggestions from Clang&rsquo;s static analyzer.</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>korli fixed a regression in the bootloader following the TSC refactoring from the previous month.</p> <p>korli adjusted the context-switch logic on x86_64 to always load the default values into FPU and SSE control registers. This actually saves two instructions compared to the old code, and fixes some kernel panics caused when applications (or drivers) set certain CPU flags.</p> <p>waddlesplash imported <code>stpncpy</code> from musl (a POSIX function we were missing.)</p> <p>waddlesplash allowed negative values to be passed to <code>release_sem_etc</code> when the <code>B_RELEASE_ALL</code> flag was set. Such values will be used as the argument to <code>thread_unblock</code>, meaning the awoken thread will receive an error code instead of <code>B_OK</code> on wakeup. (This was used in the refactored TUN/TAP driver, which needs to wake up threads waiting on <code>net_fifo</code>s without having them actually receive any buffers.)</p> <h3 id="build-system">Build system</h3> <p>A patch by KapiX to add support for checkouts created with <code>git worktree</code> was merged. (The build system needs to track the timestamp of a few files in the <code>.git</code> dir, which has to be found differently when <code>git worktree</code> is in use.)</p> <p>humdinger added the Genio IDE (a native IDE for Haiku)&rsquo;s <code>.genio</code> directory to the <code>.gitignore</code>.</p> <p>waddlesplash deleted an old file in the root of the repository containing configuration for a code scanning service that no longer exists.</p> <h3 id="documentation">Documentation</h3> <p>mmu_man added mentions of the need for <code>python3</code> and <code>zstd</code> in <code>ReadMe.Compiling.md</code>.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>kallisti5 reduced the page table tracing in the RISC-V EFI bootloader. He also added a missing page table type.</p> <h3 id="haikuports">HaikuPorts</h3> <p>The problems in QtWebEngine (Chromium) causing crashes on YouTube and many other sites were fixed by korli. We are still running on a Qt 5.x version of QtWebEngine (so, a Chromium version well behind the current, though with whatever security patches the Qt developers backported), but &ldquo;Falkon&rdquo; is a much more usable web browser now (perhaps better than GNOME Web in some respects.)</p> <p>The node.js port was bumped to the latest LTS version (and npm along with it.)</p> <p>waddlesplash pushed a new version of Xlibe, this time fixing a few problems with custom cursors (following a note on the forum about Drawterm, the Plan 9 remoting client, being affected by them.)</p> <h3 id="thats-all-folks">That&rsquo;s all, folks!</h3> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p> <p>Following the discussion on the forum after last activity report (and some others elsewhere), I&rsquo;ve made a few adjustments to my TODO list as well as the R1/beta5 milestone. Development continues&hellip;</p>Haiku Activity & Contract Report, October 2023https://www.haiku-os.org/blog/waddlesplash/2023-11-14_haiku_activity_contract_report_october_2023/waddlesplashTue, 14 Nov 2023 22:30:00 -0400https://www.haiku-os.org/blog/waddlesplash/2023-11-14_haiku_activity_contract_report_october_2023/<p>This report covers hrev57309 through hrev57363 (again a bit of a shorter month than average.)</p> <h3 id="applications">Applications</h3> <p>davidkaroly implemented DWARFv5 line-info in Debugger, allowing GCC 13-generated binaries to be debugged without needing to recompile them with specific command line flags, e.g. <code>-gdwarf-4</code>. He then proceeded to implement support for more DWARFv5 features (though there&rsquo;s still more to be done here before GCC can be allowed to generate full DWARFv5 by default without causing problems for Debugger), and fixed some typos along the way.</p> <p>jscipione fixed Deskbar submenus to all use the &ldquo;menu&rdquo; font. Previously, some of them used the &ldquo;plain&rdquo; font and some the &ldquo;menu&rdquo; font, which made things obviously inconsistent when two different fonts were in use for these; now things should be properly consistent.</p> <p>apl fixed some minor issues and improved the performance of HaikuDepot a bit, and also fixed a corner-case regarding invalid characters being entered into the username field. He also added more logging to allow for easier debugging of performance issues.</p> <p>waddlesplash fixed color name handling in Terminal, which was incorrect for non-English locales and broke adjusting colors entirely. Now things should work properly.</p> <p>waddlesplash fixed a buffer overflow causing corruption and crashes in People when very long filenames were used, and cleaned up the code a bit at the same time.</p> <p>humdinger improved layouting in BootManager, addressing some problems reported on the bugtracker.</p> <p>PulkoMandy fixed dangling reference issues with Transformer objects in <code>libicon</code>, fixing crashes in Icon-O-Matic.</p> <p>waddlesplash fixed the build of DebugAnalyzer (a tool for visualizing kernel scheduling reports.)</p> <p>nielx tweaked some UI text in mail_daemon, syslog_daemon, and Chart, upon suggestion of jt15s.</p> <p>waddlesplash fixed Debugger attaching to another application upon request of <code>debug_server</code> while already running and attached to some other application (e.g. Debugger is already open, then a crash occurs and one selects &ldquo;Debug&rdquo; at the prompt.) This had been broken for multiple years, but apparently nobody noticed before now.</p> <h3 id="command-line-tools">Command line tools</h3> <p>zdykstra removed an old <code>whence</code> command implementation from the default <code>/etc/profile</code> that was causing problems when running <code>zsh</code>.</p> <h3 id="kits">Kits</h3> <p>humdinger fixed an edge-case in <code>BSpinner</code> where the increment/decrement buttons would remain incorrectly enabled or disabled when the minimum or maximum value was changed with a value already set. jackburton79 added some missing invocations of base class methods, also to the Spinner classes.</p> <p>PulkoMandy refactored the FFmpeg media plugin to remove usages of deprecated methods. (This commit introduced a pretty bad memory leak which was only solved this month, however.)</p> <h3 id="servers">Servers</h3> <p>PulkoMandy added another Noto regional font to the app_server fallback list. (At present, characters not present in a specified font can only come from fonts in a hard-coded fallback list. Eventually this should be changed, but for now, we simply add whatever fonts are needed from the standard set into it.)</p> <h3 id="drivers">Drivers</h3> <p>X512 refactored the PCI bus manager to dynamically register host controllers. This allows multiple PCI busses/domains to be used on a single system, which is needed on some RISC-V machines (and likely also some ARM machines, but we&rsquo;re not quite there yet.)</p> <p>waddlesplash refactored the FreeBSD driver compatibility layer&rsquo;s <code>cvar</code> system to not need access to the C++ <code>ConditionVariable</code> structure size in a C-only header. This allowed for the removal of the <code>kernel_c++_structs</code> generated header system, which was used for only this one structure, and was known to cause issues with Jam dependency resolution.</p> <p>kallisti5 added a missing connector type to the <code>radeon_hd</code> modesetting driver.</p> <p>korli refactored the VirtIO subsystem to support &ldquo;modern&rdquo; devices (specification version 1.0 and following), which resolves some enhancement requests about problems running Haiku on QEMU-based virtualization that enabled this by default. He also fixed request serialization problems in <code>virtio_block</code>, fixing hangs and deadlocks when booted from that storage mechanism.</p> <h3 id="file-systems">File systems</h3> <p>phcoder fixed computation of partition sizes in UFS2, and also fixed compilation of the <code>ufs2_shell</code> test harness.</p> <p>waddlesplash (following a patch from jackburton79) fixed some preprocessor checks in the <code>fs_shell</code> for 32 vs. 64-bit pointer sizes, which weren&rsquo;t correct for non-x86 platforms and were causing build failures.</p> <p>korli made a number of fixes to the ext2/3/4 driver, resolving problems in the block-bitmap find-previous routine, directory iteration, rename incorrectly not invoking unlink when necessary, and adding missing metadata checksum recalculations.</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>puckipedia fixed a bug in the kernel thread scheduler to only unassign a thread from a core if the thread is not pinned. This fixes some KDLs occasionally encountered when disabling CPUs (though not all of them.)</p> <p>waddlesplash implemented TSC calibration via hypervisor CPUID leaves in the bootloader. At present, only the &ldquo;VMware&rdquo; mechanism is supported (however this mechanism is also implemented by QEMU/KVM, not just VMware itself.) The TSC is critical for measuring the system uptime, CPU time, sleep times, and more, and it is typically calibrated early in the boot by detecting how fast it increments in a busy-loop. However, on hypervisors (virtual machines), CPU cycles can sometimes be transparently &ldquo;stolen&rdquo; from a guest machine, causing such calibration to turn up incorrect results. So when the hypervisor itself specifies the TSC frequency directly, we can just read this instead of trying to detect it, which will be much more accurate.</p> <p>waddlesplash fixed buffer overflows in the libroot ICU timezone handling and added more error checking as well, fixing crashes in some command-line tools when too-long timezone names/paths were specified.</p> <p>korli changed the VM to always put memory pages into the clear page queue in-order. Previously they were added in reverse order, which wasn&rsquo;t technically a problem but caused inefficiencies elsewhere (as it meant page allocations across the whole system were typically not in order, unless they were specifically requested to be.) This could (among other things) improve DMA performance by making bounce-buffers unnecessary in more cases.</p> <p>korli made a first round of fixes to x86_64 context switch handling for floating-point state, to fix some rare math-exception KDLs.</p> <h3 id="build-system">Build system</h3> <p>waddlesplash (based on a patch from X512) adjusted the Jam header rules to only add C++ header directories to the command line when compiling C++ files. (This fixes problems encountered with some GCC headers that have two separate versions for C and C++.)</p> <p>kallisti5 adjusted the file download process to automatically retry a bit harder when downloads during a build are interrupted (e.g. by bad internet connections.)</p> <p>PulkoMandy enabled <code>-Werror</code> for use of deprecated functions.</p> <p>nielx updated the Debian image used for the Docker cross-compiler image.</p> <p>kallisti5 made some improvements to the Google Cloud Platform preparation script (i.e. for running Haiku inside Google Cloud VMs.)</p> <h3 id="documentation">Documentation</h3> <p>PulkoMandy removed an obsolete comment from the MIDI kit.</p> <h3 id="arm--risc-v-and-ppc">ARM &amp; RISC-V (and PPC?)</h3> <p>Yn0ga (a new contributor!) submitted a few patches dusting off the basics of Haiku&rsquo;s PowerPC port, removing things that are no longer needed, adjusting spec files and flags, adding a few hook functions to the bootloader, and more.</p> <p>X512 and kallisti5 bumped the RISC-V port up to GCC 13.</p> <h3 id="haikuports">HaikuPorts</h3> <p>There was a healthy amount of activity at HaikuPorts last month, with updates to quite a lot of ports, including Qt Creator, Python, FFmpeg, Wireshark, Git, LibreOffice, Node.JS (now the latest LTS version), and lots more, as well as quite a lot of entirely new ports of applications, games, tools, and libraries.</p> <h3 id="xlibe">Xlibe</h3> <p>I (waddlesplash) spent a few days working on parts of Xlibe, the X11/Xlib compatibility layer. Earlier in the year I&rsquo;d implemented a few features needed by FLTK (which is now packaged in HaikuDepot and being used by a few ports), and last month I was CC&rsquo;ed on the GNUplot port, which built against Xlibe but then always showed a blank window. I fixed the problems causing that, as well as some others surrounding GNUplot and also Conky, and so both of those should be able to be built against Xlibe to get a graphical environment now.</p> <p>After that, I also took another look at Tcl/Tk, to see if I could fix the remaining problems and make that work, as there were some more requests for Python Tkinter (which doesn&rsquo;t work with the SDL2-based Tk) or simply a Tk that works without all the limitations of the SDL2-based one for <code>git-gui</code> and other such ports. I fixed a number of minor problems with the port (enabling the Tk automated testsuite to start and function), and based on some clues left the month before by X512 and PulkoMandy, managed to get a handle on the source of the drawing glitches in Tk running under Xlibe: it&rsquo;s related to overlapping sibling BViews.</p> <p>Haiku allows child views to overlap parts of the parent (obviously) but it doesn&rsquo;t do anything about sibling views that overlap (and neither did BeOS.) Tk, however, seems to presume this is fully legal and does it quite frequently for drawing backgrounds. On other OSes, X11 supports it natively, Win32 supports it if a certain flag is enabled, and macOS doesn&rsquo;t support it at all (for which Tk has a platform-specific workaround in which it manually sets clipping regions for all areas.)</p> <p>There was some subsequent discussion about whether or not to permit this in Haiku and thus give app_server defined behaviors when it happens (instead of behaving strangely and unpredictably), but it wasn&rsquo;t conclusive. I don&rsquo;t know if it makes sense to spend much time on it for one specific ported X11 application. More likely it makes sense to patch Tk, like is done for macOS, but I don&rsquo;t think that&rsquo;d be a productive use of my time at the moment. Perhaps someone else motivated will eventually look into the problem (and I can of course give you pointers as to what I&rsquo;ve tried already, both inside Tk and inside Xlibe.)</p> <h3 id="thats-all-folks">That&rsquo;s all, folks!</h3> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p> <p>There&rsquo;s starting to be talk about making another release, though nothing definite (and there&rsquo;s still tickets in the milestone, and potentially more tickets that should be added there.) So: What do you think belongs in the next Haiku release? Most of the changes over the past year since the previous release were relatively &ldquo;non-flashy&rdquo; (this report is a pretty good example of that), but make real contributions to the system&rsquo;s stability and usability nonetheless. What problems (beyond large and well-known missing features besides &ldquo;3D acceleration&rdquo;) have you encountered running Haiku recently that stop you from using it more often?</p>Haiku Developer Tools Update (October 2023)https://www.haiku-os.org/blog/nielx/2023-10-28_haiku_developer_tools_update_october_2023/nielxSat, 28 Oct 2023 10:35:19 +0100https://www.haiku-os.org/blog/nielx/2023-10-28_haiku_developer_tools_update_october_2023/<p>This article lists some of the recent updates for developer tools that are available in the HaikuPorts repository. Many of these can be installed from HaikuDepot.</p> <p>This is the first article on this topic, but it may become a regular (quarterly?) series. Let me know in the comments what you find of these notes, and what else you think should be covered.</p> <h2 id="gcc-1320">GCC 13.2.0</h2> <p>The Haiku nightlies are now built with GCC 13.2.0 as the primary compiler (the previous version was GCC 11.3.0) and it is available for software developers. It can also be used on a Haiku R1 beta 4 system. The packages support <strong>C, C++, and Fortran.</strong></p> <p>The updated version includes all improvements in the <a href="https://gcc.gnu.org/gcc-12/changes.html">GCC 12</a> and <a href="https://gcc.gnu.org/gcc-13/changes.html">GCC 13</a> builds. Since Haiku&rsquo;s primary language is C++, it is particularly notable that there is now improved support for C++20 features, as well as some more early support for C++23.</p> <p>Check out the packages on <strong>HaikuDepot</strong>:</p> <ul> <li> <p><code>gcc</code>: <a href="https://depot.haiku-os.org/#!/pkg/gcc/haikuports/haikuports_x86_64/13/2/0_2023_08_10/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/gcc_x86/haikuports/haikuports_x86_gcc2/13/2/0_2023_08_10/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>gcc_fortran</code>: <a href="https://depot.haiku-os.org/#!/pkg/gcc_fortran/haikuports/haikuports_x86_64/13/2/0_2023_08_10/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/gcc_x86_fortran/haikuports/haikuports_x86_gcc2/13/2/0_2023_08_10/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>gcc_jit</code> (new!): <a href="https://depot.haiku-os.org/#!/pkg/gcc_jit/haikuports/haikuports_x86_64/13/2/0_2023_08_10/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/gcc_x86_jit/haikuports/haikuports_x86_gcc2/13/2/0_2023_08_10/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>gcc_syslibs</code>: <a href="https://depot.haiku-os.org/#!/pkg/gcc_syslibs/haikuports/haikuports_x86_64/13/2/0_2023_08_10/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/gcc_x86_syslibs/haikuports/haikuports_x86_gcc2/13/2/0_2023_08_10/-/3/x86_gcc2">32 bit</a></p> </li> </ul> <h2 id="binutils-241">Binutils 2.41</h2> <p>Binutils is a collection of tools to handle binaries, best known for the <code>ld</code> linker. Recently, binutils was updated to the 2.41 release. The previous version of binutils was 2.31.</p> <p>Check the binutils package on <strong>HaikuDepot</strong>: <a href="https://depot.haiku-os.org/#!/pkg/binutils/haikuports/haikuports_x86_64/2/41/-/-/1/x86_64?bcguid=bc281-YHHL">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/binutils_x86/haikuports/haikuports_x86_gcc2/2/41/-/-/1/x86_gcc2?bcguid=bc358-QDCL">32 bit</a></p> <h2 id="llvm-17">LLVM 17</h2> <p>The LLVM Compiler Infrastructure Project provides libraries that power many modern languages, like Rust. It also ships with a C/C++ compiler called <code>clang</code> and the <code>clangd</code> package is the standard LSP-server for C/C++ used by many IDEs, including Qt Creator (see below). The package was recently updated to the <a href="https://discourse.llvm.org/t/llvm-17-0-1-released/73549">17.0.1</a> release.</p> <p>Check out the packages on <strong>HaikuDepot</strong>:</p> <ul> <li> <p><code>llvm17</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_python310/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">32 bit</a></p> </li> <li> <p><code>llvm17_clang</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_clang/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_clang/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>llvm17_clang_analysis</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_clang_analysis/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_clang_analysis/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>llvm17_libs</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_libs/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_libs/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>llvm17_libunwind</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_libunwind/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_libunwind/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>llvm17_lld</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_lld/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_lld/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> <li> <p><code>llvm17_python310</code>: <a href="https://depot.haiku-os.org/#!/pkg/llvm17_python310/haikuports/haikuports_x86_64/17/0/1/-/3/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/llvm17_x86_python310/haikuports/haikuports_x86_gcc2/17/0/1/-/3/x86_gcc2">32 bit</a></p> </li> </ul> <h2 id="python-310-311--312">Python 3.10, 3.11 &amp; 3.12</h2> <p>The Python interpreter has been updated recently as well, and the standard python interpreter on Haiku is now Python 3.10.</p> <p>Check out the packages on <strong>HaikuDepot</strong>:</p> <ul> <li> <p><code>python3.10</code>: <a href="https://depot.haiku-os.org/#!/pkg/python3.10/haikuports/haikuports_x86_64/3/10/13/-/1/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/python3.10_x86/haikuports/haikuports_x86_gcc2/3/10/13/-/1/x86_gcc2">32 bit</a></p> </li> <li> <p><code>python 3.11</code>: <a href="https://depot.haiku-os.org/#!/pkg/python3.11/haikuports/haikuports_x86_64/3/11/6/-/1/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/python3.11_x86/haikuports/haikuports_x86_gcc2/3/11/6/-/1/x86_gcc2">32 bit</a></p> </li> <li> <p><code>python 3.12</code>: <a href="https://depot.haiku-os.org/#!/pkg/python3.12/haikuports/haikuports_x86_64/3/12/0/-/1/x86_64?bcguid=bc2938-FWTK">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/python3.12_x86/haikuports/haikuports_x86_gcc2/3/12/0/-/1/x86_gcc2">32 bit</a></p> </li> </ul> <h2 id="qt-creator-1103">Qt Creator 11.0.3</h2> <p>This IDE from the makers of Qt supports full-fledged C++ development, including code completion using <code>clangd</code> (part of the LLVM package). See what&rsquo;s new in <a href="https://www.qt.io/blog/qt-creator-11-released">version 11</a>. Previous release on Haiku was 6.0.1.</p> <p>Check out the package on <strong>HaikuDepot</strong>: <a href="https://depot.haiku-os.org/#!/pkg/qt_creator/haikuports/haikuports_x86_64/11/0/3/-/2/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/qt_creator_x86/haikuports/haikuports_x86_gcc2/11/0/3/-/2/x86_gcc2">32 bit</a></p> <h2 id="cudatext-120000">CudaText 1.200.0.0</h2> <p>This modern text editor for developers is written in Object Pascal, is fast, and has a lot of features that are available as plugins, including code completion using <code>clangd</code> (part of the LLVM package). See the <a href="https://github.com/Alexey-T/CudaText/blob/1.200.0/app/readme/history.txt">change log</a> to see what&rsquo;s new.</p> <p>Check out the package on <strong>HaikuDepot</strong>: <a href="https://depot.haiku-os.org/#!/pkg/cudatext/haikuports/haikuports_x86_64/1/200/0.0/-/1/x86_64?bcguid=bc2503-PXKT">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/cudatext_x86/haikuports/haikuports_x86_gcc2/1/200/0.0/-/1/x86_gcc2">32 bit</a></p> <h2 id="haiku-format-17">haiku-format 17</h2> <p>This tool is a fork of the <code>clang-format</code> tool that is part of the LLVM package (see above). It features a code formatting that covers (almost all of) Haiku&rsquo;s coding style. It is beneficial to run this on your code changes before submitting it to Haiku. It has recently been updated with the latest changes in <code>clang-format</code>, and it includes some further refinements for the coding style. See the <a href="https://discuss.haiku-os.org/t/haiku-coding-guidelines-and-haiku-format/14047/1">announcement</a> on our forums.</p> <p>Check out the package on <strong>HaikuDepot</strong>: <a href="https://depot.haiku-os.org/#!/pkg/haiku_format/haikuports/haikuports_x86_64/17/0/1/-/2/x86_64">64 bit</a> | <a href="https://depot.haiku-os.org/#!/pkg/haiku_format_x86/haikuports/haikuports_x86_gcc2/17/0/1/-/2/x86_gcc2?bcguid=bc3071-UTWU">32 bit</a></p> <h2 id="dotnet-gsoc-port">DotNet GSOC Port</h2> <p>One of the Google Summer of Code programs this year was the porting of <code>dotnet</code> to Haiku. The student continued on previous work and touched a lot of code, including in the Haiku kernel itself. The <a href="https://www.haiku-os.org/blog/trungnt2910/2023-08-20_gsoc_2023_dotnet_port_final_report/">final blog post</a> is a very interesting read!</p>Haiku Activity & Contract Report, September 2023https://www.haiku-os.org/blog/waddlesplash/2023-10-12_haiku_activity_contract_report_september_2023/waddlesplashThu, 12 Oct 2023 21:30:00 -0400https://www.haiku-os.org/blog/waddlesplash/2023-10-12_haiku_activity_contract_report_september_2023/<p>This report covers hrev57257 through hrev57308.</p> <p>This was a bit of a shorter month than usual (for me, at least.)</p> <h3 id="applications">Applications</h3> <p>Zardshard cleaned up some code in Icon-O-Matic, removing <code>using namespace</code> directives from headers. He also fixed building Deskbar with <code>ASSERT</code>s compiled in.</p> <p>humdinger adjusted the &ldquo;low battery&rdquo; notifications in PowerStatus allow setting distinct sounds for &ldquo;battery low&rdquo; and &ldquo;battery critical&rdquo; notifications.</p> <p>davidkaroly did a bunch of work on Debugger to allow it to (partially) parse newer versions of the DWARF debugging information standard, as GCC now generates.</p> <p>jacereda (a new contributor!) refactored Keymap&rsquo;s &ldquo;Modifier keys&rdquo; dialog, and added support for remapping the &ldquo;Caps Lock&rdquo; key (in addition to the already-remappable Shift, Control, Option, and Command keys.)</p> <p>humdinger fixed some internationalization logic in the &ldquo;Colors&rdquo; preferences window in Terminal.</p> <p>waddlesplash fixed Tracker file panels having their &ldquo;Open&rdquo; buttons incorrectly enabled, or incorrectly allowing files to be double-clicked for opening, when only directories were supposed to be openable. (This fixed a bug dating all the way back to the original OpenTracker source import in 2001!)</p> <p>apl implemented the basics of token-based authentication (part of Haiku&rsquo;s ongoing switch to a &ldquo;single-sign-on&rdquo; system instead of having separate account databases for all our online services) in HaikuDepot.</p> <p>humdinger added a setting to limit FPS in GLTeapot. Previously, if the running display driver supported detecting when retraces occurred, GLTeapot would aways wait for it and thus only produce at most as many FPS as the screen refresh rate, making it not very useful as the &ldquo;benchmark&rdquo; it has been seen as in the past. Now this can be turned on and off, allowing one to get hundreds to thousands of FPS once more.</p> <h3 id="command-line-tools">Command line tools</h3> <p>cpr420 fixed the <code>filteredquery</code> command&rsquo;s displaying of results, argument parsing, and cleaned up the code somewhat.</p> <h3 id="kits">Kits</h3> <p>waddlesplash fixed the logic of <code>BBufferedDataIO::Write</code>, which was quite incorrect (but apparently not used anywhere, whether inside Haiku&rsquo;s own source code or in third-party software) and added some unit tests for it.</p> <h3 id="drivers">Drivers</h3> <p>waddlesplash fixed a NULL-dereference panic in the IPv4 networking module that could happen in certain scenarios involving multicast.</p> <p>waddlesplash synchronized the <code>iaxwifi200</code> (aka. <code>iwx</code>) driver with OpenBSD, bringing in a lot of changes that utilize newer firmware versions.</p> <h3 id="file-systems">File systems</h3> <p>waddlesplash fixed some locking problems in packagefs causing whole-system hangs which had been uncovered while testing certain experimental (not yet enabled) HaikuPorts recipes. (The packages produced still can&rsquo;t be installed properly, but at least the failure will now be a graceful one.)</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>trungnt2910 added a definition of <code>static_assert</code> for C11 and up. (For C++11, this is a compiler builtin, but for C11 it needs to be defined in a header file.)</p> <p>davidkaroly adjusted some of the internal musl headers and by bringing in changes from upstream, fixing build issues on non-x86 architectures.</p> <p>waddlesplash deleted some old and long-unused private headers, and updated some of the BSD compatibility headers to newer versions from FreeBSD.</p> <p>waddlesplash fixed a type mixup causing kernel panics in the XSI (POSIX extensions) message queue implementations, and performed a number of code cleanups to it while doing so.</p> <p>waddlesplash altered the &ldquo;trap in kernel debugger&rdquo; loop to invoke the <code>PAUSE</code> (or equivalent) instruction. This loop runs on all CPUs <em>besides</em> the one actively running the kernel debugger, and so it spends most of its time waiting for something to happen. Invoking <code>PAUSE</code> just like waiting spinlocks do hints to the CPU that we are in a busy-loop and to avoid consuming as much power. (Even in a VM with only a few cores allocated, the lessened power consumption and thus difference in fan spin-up are very noticeable.)</p> <p>waddlesplash removed the custom implementations of <code>arch_debug_get_caller</code> and replaced them with a compiler builtin.</p> <p>waddlesplash rewrote the <code>B_DEBUG_SPINLOCK_CONTENTION</code> debugging option. This option (disabled by default through a <code>#define</code>; the whole system must be specially recompiled if it is enabled) allows measuring how much time is spent holding or waiting for spinlocks across the system. As spinlocks are always dealt with in interrupts-disabled mode, it&rsquo;s thus useful for tracking down performance problems or hotspots in areas the kernel profiler can&rsquo;t see.</p> <p>puckipedia fixed stack alignment in <code>arch_debug_call_with_fault_handler</code>, used in the kernel debugger for invoking various commands. They also fixed a bug in the kernel&rsquo;s x86(_64)-specific code to send inter-CPU interrupts, which was causing incorrect or missed deliveries (fixing problems with Haiku running on machines large numbers of cores randomly stalling or hanging.)</p> <h3 id="build-system">Build system</h3> <p>waddlesplash disabled autovectorization optimizations for the kernel, at least for the time being. These compiler optimizations turn regular operations into SIMD operations (e.g. using SSE2 instructions on x86, or NEON instructions on ARM), which can sometimes provide significant performance improvements but requires use of the FPU (among other limitations.) There should be no reason these can&rsquo;t be used in the kernel, and indeed we&rsquo;ve had them enabled for years, but after the GCC 13 upgrade, they were identified as the cause of various problems running Haiku in virtual machines (and not just in one virtualization platform; QEMU/KVM, VMware, and VirtualBox are all affected, though in different ways.) None of the problems have thus far been observed on bare metal, but until the cause can be properly tracked down, autovectorization was disabled entirely for the kernel (it remains enabled for userland, of course.) Perhaps notably, Linux and the BSDs have much more severe restrictions on the use of the FPU (thus including SSE2, NEON, etc. instructions) in kernel-mode than Haiku does, so there&rsquo;s basically no chance for such optimizations to be done in kernel mode at all for them, and thus it&rsquo;s possible VM software has not been much tested for the use of such CPU instructions in kernel mode.</p> <h3 id="documentation">Documentation</h3> <p>Zardshard wrote proper documentation in the Haiku Book for <code>BMenu::AddDynamicItem</code>.</p> <p>PulkoMandy added a note about calling <code>GetMouse()</code> inside a <code>MouseUp()</code> hook in the <code>BView</code> documentation.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>davidkaroly dusted off some of the M68K (!) port code and brought it partially in line with some of the changes that have occurred in the years since it was last touched, specifically in the bootloaders, musl internal headers, kernel architecture-specific files, and other related areas. He also performed similar cleanups to the PPC port.</p> <p>davidkaroly silenced some warnings in the ARM64 port.</p> <p>X512 added another RISCV-specific section to the ELF loaders (just ignored for now as it&rsquo;s not needed at the moment.) He also refactored page-table flags handling to have more readable code and be properly atomic, fixing some concurrent access bugs.</p> <h3 id="haikuports">HaikuPorts</h3> <p>LLVM/Clang 17 packages are now available, with a lot of Haiku-specific fixes and patches compared to prior versions.</p> <p>Running of Fortran applications compiled with GCC 13&rsquo;s Fortran compiler was fixed.</p> <h3 id="thats-all-folks">That&rsquo;s all, folks!</h3> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p>Haiku Activity & Contract Report, August 2023https://www.haiku-os.org/blog/waddlesplash/2023-09-12_haiku_activity_contract_report_august_2023/waddlesplashTue, 12 Sep 2023 21:30:00 -0400https://www.haiku-os.org/blog/waddlesplash/2023-09-12_haiku_activity_contract_report_august_2023/<p>This report covers hrev57184 through hrev57256.</p> <p>It&rsquo;s worth noting: the main Haiku CI is currently offline as the developer who was hosting the build machine moved to a location with much slower internet. A new build machine and home for the CI has already been selected, but isn&rsquo;t fully online yet, so the nightly builds are a bit behind at the moment.</p> <h3 id="applications">Applications</h3> <p>Zardshard contributed some changes to refactor parts of Debugger&rsquo;s CLI event handling, especially the <code>WaitForThreadOrUser</code> routine and also the message-passing facilities.</p> <p>PulkoMandy implemented clipboard paste in SerialConnect.</p> <p>Humdinger adjusted Tracker&rsquo;s logic that converts a size in bytes to a user-friendly string, and added a comment for translators at the same time.</p> <p>Humdinger added a mime-db entry for email drafts (which have a different type than regular email files.)</p> <p>Zardshard (a GSoC &lsquo;23 participant) made the left sidebar&rsquo;s lists resizable using splitters, fixed a window resizing bug, improved various saving-related error messages, and fixed some bugs that occurred when pasting text. Zardshard also implemented perspective transformations for Icon-O-Matic and HVIF, a long-requested and pretty substantial change, which required importing newer AGG rendering code. Finally, he made sidebar list items copy/pastable, and fixed selection of multiple path vertices.</p> <p>Humdinger improved CodyCam&rsquo;s icon and added its missing Icon-O-Matic source file to the repository.</p> <p>waddlesplash fixed some screen vendor names used in Screen preferences to remove extraneous content, or to restore proper names that had been deleted from the upstream database the list was sourced from. (This fixes the scary &ldquo;DO NOT USE&rdquo; messages that would sometimes appear in Screen preferences.)</p> <p>waddlesplash adjusted Debugger&rsquo;s debug-info loading code to print a more intelligible warning when encountering unsupported DWARF versions.</p> <p>waddlesplash reworked ShowImage to use layouts for the main view area. (This fixes an incorrect sizing problem that would happen when hiding or showing the toolbar.)</p> <p>waddlesplash refactored Tracker&rsquo;s <code>NavMenu</code> and <code>SlowContextPopup</code> classes, first reducing the differences between them, and then folding the latter into the former (as they were near-duplicates of each other.) This duplication dated all the way back to the first OpenTracker source commit over 20 years ago, and probably long before that! He also merged some closely related methods in <code>BPoseView</code> that had TODO comments indicating they should be combined.</p> <p>davidkaroly implemented DWARFv4 line-info parsing in Debugger. (However, our new GCC emits v5 by default, despite some flags set internally that should prevent this, so one still needs to specify <code>-gdwarf-4</code> in order to get GCC to output debug-info parseable by our Debugger.)</p> <p>waddlesplash fixed Zip-O-Matic&rsquo;s logic to select the newly-created zip file in Tracker.</p> <h3 id="command-line-tools">Command line tools</h3> <p>PulkoMandy moved the &ldquo;mountpoint&rdquo; column last in the <code>df</code> command&rsquo;s output, making the table much more readable when there are long mountpoint paths.</p> <h3 id="kits">Kits</h3> <p>PulkoMandy made some improvements to <code>BTextView</code>&rsquo;s internal height-vs-width metrics calculation routine when the text view is in read-only mode.</p> <p>waddlesplash adjusted lock acquisition in <code>BMenu::AddItem</code>, which should hopefully fix some of the long-standing crashes known to happen in the Network preflet, among others.</p> <p>waddlesplash implemented <code>getentropy</code> (through the &ldquo;generic syscall&rdquo; mechanism; it merely reads <code>/dev/{u}random</code> without needing a file descriptor) and then merged an <code>arc4random</code> implementation to go with it.</p> <p>waddlesplash synchronized the DNS resolver up to NetBSD 9.3. Previously it was on an older version of the separately released &ldquo;netresolv&rdquo; code, but now it uses NetBSD&rsquo;s mainline source directly (with a handful of Haiku-specific patches.) This resolved some tickets related to DNS resolution.</p> <h3 id="drivers">Drivers</h3> <p>korli added support for some newer hardware generations to the PCH &ldquo;thermal&rdquo; driver (used on Intel machines.)</p> <h3 id="file-systems">File systems</h3> <p>phcoder contributed a large number of fixes to the UFS2 filesystem driver, including to directory iteration, read support, implementing <code>access()</code>, and a lot more. The much improved driver has been added to the default builds, so Haiku can now read (but not yet write) UFS2 filesystems.</p> <p>phcoder implemented the <code>truncate</code> operation in the <code>fs_shell</code>&rsquo;s FUSE layer, making the <code>bfs_fuse</code> driver much more usable on Linux (and other OSes too.)</p> <p>waddlesplash rewrote <code>ramfs</code>&rsquo;s Query parser/evaluator. Previously, it used its own (much outdated) copy of the one from BFS; now it uses the &ldquo;common&rdquo; one (which was created during <code>packagefs</code> development, but until now had not been used by anything else but <code>packagefs</code> itself.) This resulted in a diff with a net deletion of over 1500 lines. He also adjusted <code>ramfs</code> to not show up in certain Tracker views (it was hidden from a few already, but still appeared in others.)</p> <h3 id="libroot--kernel">libroot &amp; kernel</h3> <p>waddlesplash implemented an &ldquo;errata patch&rdquo; for the recently-disclosed AMD &ldquo;Zenbleed&rdquo; vulnerability.</p> <p>trungnt2910 fixed the &ldquo;connected&rdquo; status reporting of UNIX domain sockets so that operations like <code>getpeername</code> work correctly. waddlesplash fixed the UNIX domain socket <code>Close</code> routines as well as the read/write routines&rsquo; checking of the shutdown status, and korli implemented <code>MSG_DONTWAIT</code> for them. trungnt2910 then implemented <code>SO_RCVBUF</code> for all kinds of UNIX sockets.</p> <p>waddlesplash removed and refactored some functions in kernel utility headers introduced in the lead-up to the <code>event_queue</code> implementation which turned out to not be needed after all, and ported some assertions and fault-tolerance between <code>DoublyLinkedList</code> and <code>DoublyLinkedQueue</code>.</p> <p>waddlesplash fixed some issues uncovered or introduced with the <code>event_queue</code>/<code>kqueue</code> changes the previous month, including incorrect locking around deselect operations in the kernel VFS layer, incorrect atomic flag modifications in the event-queue system itself, potential (or actual) use-after-frees in the older select code, and more.</p> <p>waddlesplash adjusted the return values of <code>ConditionVariable::Notify{All}</code> and <code>ConditionVariableEntry::Wait</code> to guarantee that the number returned by <code>Notify</code> is the actual number of <code>ConditionVariableEntry::Wait</code>s that returned with the passed status. (This is needed to avoid races into deadlocks in the user-mutex system.)</p> <p>waddlesplash patched libroot&rsquo;s default allocator to acquire and then release or reinitialize another lock after fork, to reduce deadlocks.</p> <p>waddlesplash silenced a warning from the bootloader&rsquo;s ELF loader when encountering <code>PT_EH_FRAME</code>, which the new GCC inserts by default.</p> <p>korli tweaked the Intel microcode loading logic to not load microcode when it&rsquo;s already up to date.</p> <p>axeld added a missing routine (<code>user_memset</code>) to the <code>kernelland_emu</code> library, used for testing kernel components in userland.</p> <p>jmairboeck added a missing newline to a print in the kernel debugger.</p> <p>korli added a <code>sched_getcpu</code> routine to <code>libgnu</code>. This is a GNU extension, but can be used to get the ID of the CPU that a thread is currently running on (which can of course change at any time.) On newer x86_64 CPUs, it uses a CPU feature to read the ID in userland; on others, it uses a syscall for this purpose.</p> <p>waddlesplash adjusted some feature-flags in headers to use the standard forms from <code>features.h</code>.</p> <p>waddlesplash fixed the indentation of some <code>locale_t</code> functions, and then added an internal method to expose the &ldquo;POSIX&rdquo;/&ldquo;C&rdquo; locale directly. He then replaced a number of time-related C library methods with the implementations from <code>musl</code> (replacing FreeBSD, or in some cases much older, code.) He also synchronized a number of other files with FreeBSD, pulling in updates and bugfixes.</p> <h3 id="build-system">Build system</h3> <p>madmax removed accidental trigraphs from various parts of the build. (These are character sequences in source code starting with an unescaped <code>?</code> used for encoding various characters. The fix is just to add a <code>\</code> in front of the <code>?</code>.)</p> <p>nielx updated the repository of packages (from HaikuPorts) used in the building of Haiku images.</p> <p>PulkoMandy removed <code>-Wno-error=deprecated</code> from the global compiler flags, as it&rsquo;s no longer needed. (Note that this is different from <code>-Wno-error=deprecated-declarations</code>, which we still need.) He also removed some usage of deprecated C++ features in a few files across different components.</p> <p>tqh made some minor edits to the code here and there to silence some warnings.</p> <p>waddlesplash fixed the type of <code>MAIL:draft</code> and <code>MAIL:flags</code> filesystem indexes in the default install, which fixes the &ldquo;Open Drafts&rdquo; option in Mail not working properly.</p> <p>nielx fixed the build of the <code>DateFormatTest</code>.</p> <h3 id="documentation">Documentation</h3> <p>nielx updated the developer (internals) documentation on generating the package repository used to build Haiku itself, and also that for upgrading the GCC buildtools.</p> <h3 id="arm--risc-v">ARM &amp; RISC-V</h3> <p>davidkaroly updated the &ldquo;bootstrap&rdquo; package versions (which refer to those in <code>HaikuPortsCross</code>) for <code>arm</code>, <code>arm64</code>, <code>riscv</code>, and even <code>x86_64</code>.</p> <h3 id="haikuports">HaikuPorts</h3> <p>waddlesplash implemented <code>GraphicsExpose</code> in <code>Xlibe</code>, which fixes all scrolling in FLTK applications (previously it would just hang the entire application forever.)</p> <h3 id="website">Website</h3> <p>There are GSoC final reports out now, from <a href="https://www.haiku-os.org/blog/zardshard/">Zardshard</a> on Icon-O-Matic, and <a href="https://www.haiku-os.org/blog/trungnt2910/">trungnt2910</a> on porting .NET.</p> <h3 id="thats-all-folks">That&rsquo;s all, folks!</h3> <p>Thanks again to all who contribute to Haiku, and especially those donors who make my contract possible!</p>[GSoC 2023] VPN Support Project Final Reporthttps://www.haiku-os.org/blog/pairisto/2023-09-10_gsoc_2023_vpn_support_project_final_report/pairistoSun, 10 Sep 2023 23:29:07 -0500https://www.haiku-os.org/blog/pairisto/2023-09-10_gsoc_2023_vpn_support_project_final_report/<h2 id="project-overview">Project Overview</h2> <p>This project, undertaken as part of Google Summer of Code 2023, sets out to implement a robust TUN/TAP driver for Haiku given the increasing demand from the community for a virtual network kernel interface. This project allows for VPN software and other network-related utilities that work with TUN/TAP driver to operate seamlessly on Haiku. Throughout the project&rsquo;s duration, I&rsquo;ve documented the rationale behind key design decisions, and offered insights on the interplay between the TUN/TAP driver and Haiku&rsquo;s unique networking architecture in my series of seven progress blog posts. For those interested in trying out the TUN/TAP driver on Haiku (when the code is merged, more on that in a different section), you can use the port of OpenVPN that was made on haikuports <a href="https://github.com/haikuports/haikuports/tree/master/net-vpn/openvpn">here</a>. One last thing to say is that I could not get the TUN interface to work properly due to a lack of Point-to-Point being supported by Haiku but TAP works for both the interface and driver.</p> <h2 id="what-was-done">What Was Done</h2> <ol> <li>Added the TUN/TAP driver and interface are in a state that works for tunneled networking communications over TAP (explained why TUN can&rsquo;t be used in the next section).</li> <li>OpenVPN was ported over and modified to work on Haiku to work with the TUN/TAP driver created.</li> </ol> <h2 id="whats-left-to-do">What&rsquo;s Left To Do</h2> <ol> <li>Add dynamic driver loading so that we do not have a static devfs entry for the driver.</li> <li>Finish the Point-to-Point Protocol implementation for the OS to get the TUN functionality for the interface working as there is no way to do the routing needed for TUN to work properly with an application.</li> <li>Add a separate function for ethernet bridging for the driver.</li> </ol> <h2 id="what-have-i-learned">What Have I Learned</h2> <p>This summer taught me way more about programming concepts that I never really encountered before like mutexes, semaphores, condition variables, networking, driver development, and even how to make my workflow more efficient and so much more than I thought it was ever going to teach me.</p> <p>Kernel and system software development have been central to my project as it involved delving deep into thread safety mechanisms, how to write implementations of the common driver I/O functions for a driver, and understanding the intricacies of how an operating system&rsquo;s network stack functions and subsequently designing an interface to complement it.</p> <p>On the debugging front, I acquired significant insights into addressing kernel panics and anomalies in the driver and interface behavior. This was achieved through innovative use of tools like <code>ghidra</code> and methods such as inserting debugging statements, especially given the absence of a robust debugger in our environment.</p> <p>Another pivotal aspect of my experience was navigating and analyzing undocumented code, primarily from OpenVPN for application testing and from Haiku for driver development. This analytical process not only aided in problem-solving but also inspired me to pen down seven blog posts, capturing and documenting the solutions I discerned.</p> <h2 id="code-submitted">Code Submitted</h2> <p>My main TUN/TAP driver/interface commit has yet to be pushed upstream due to waiting on feedback from one of my reviewers but from what I can tell it is mostly ready to be pushed. the commits that I have made are:</p> <ol> <li><a href="https://review.haiku-os.org/c/haiku/+/6608/">Main Driver/Interface Commit</a> <ol> <li><a href="https://review.haiku-os.org/c/haiku/+/6898/2?usp=related-change">Indirect Ancestor to add to build definitions</a></li> </ol> </li> <li><a href="https://review.haiku-os.org/c/haiku/+/6904/1?usp=related-change">WIP Documentation for the Driver and Interface</a> <ol> <li>Here is where you can find the documentation for the driver along with how to use it via OpenVPN though it is still in a heavy work in progress</li> </ol> </li> </ol> <h2 id="usage">Usage</h2> <p>If you want to use my code now you can get it <a href="https://haiku.movingborders.es/testbuild/I19f37ef75250526d7a7f9e254fd0bf3693582c92/2/hrev57265/">here</a> as it&rsquo;s not pushed to the nightlies yet, but whenever it does get pushed in the future then one of the best ways that I have found to test it is using OpenVPN. You can installing it by using <code>pkgman install openvpn</code> and once installed you can navigate to this link <a href="https://openvpn.net/community-resources/how-to/#setting-up-your-own-certificate-authority-ca-and-generating-certificates-and-keys-for-an-openvpn-server-and-multiple-clients">here</a> from OpenVPN to help with setting up the environment. You can do either a Haiku to Haiku connection or Haiku to Linux connection, all up to you.</p> <h2 id="acknowledgements">Acknowledgements</h2> <p>Greetings once more, everyone! I&rsquo;ve now reached the final evaluation of my GSoC Project, which centered on adding VPN support to Haiku. I&rsquo;m deeply appreciative of this unique opportunity and the incredible support from this vibrant community that made tackling complex issues much easier thanks to your collective insights. A big thank you to my mentors, scottmc and korli, who played pivotal roles in my learning journey. Additionally, I want to acknowledge community members augiedoggie, pulkomandy, Begasus, x512, OscarL, axeld, and Waddlesplash for sharing their invaluable knowledge whenever I encountered challenges. Whether it was on the IRC or Mailing List, I still would not have been able to do it without all of you.</p>[GSoC 2023] VPN Support Project Update #7https://www.haiku-os.org/blog/pairisto/2023-08-31_gsoc_2023_vpn_support_project_update_7/pairistoThu, 31 Aug 2023 20:10:19 -0500https://www.haiku-os.org/blog/pairisto/2023-08-31_gsoc_2023_vpn_support_project_update_7/<h2 id="where-we-last-left-off">Where We Last Left Off</h2> <p>Last post, I left off on the problem where the select functionality was working but there are some problems as it works but not well as the average latency is above 2000ms and when using ping it drops more than 60% of packets on average. For two weeks I was working on this issue but I couldn&rsquo;t figure out what was wrong with select and given that I was coming up on the deadline of my project, I decided to go with a condition variable approach when reading data from the driver for both the application and interface side. For the application side, it does have a timeout on it so that write can also take place since OpenVPN uses select/poll which will check if both read and write can happen at the same time, and since I am blocking one of them, it would just infinitely block both operations until read was fine. While select functionality is something that should be implemented with the driver, timing was not on my side with this issue so for those who want to take more of a crack at finding out what the problem is, here was how I tested it:</p> <ul> <li>The main way I tested this issue was by setting up a OpenVPN client on Haiku and then trying to connect to a server instance of OpenVPN on Linux (since this select issue also happens when you run a server instance of OpenVPN on Haiku) <ul> <li>From there, I use ping on both the client and server side to ping each other</li> </ul> </li> <li>Another test to run if you don&rsquo;t have the ability to do the prior you can do <code>./openvpn --remote any.ip.addr.here --verb 9 --ping 1 --dev tap0</code></li> <li>There are also the unit tests found in the OpenVPN repo <a href="https://github.com/OpenVPN/openvpn/tree/master/tests">here</a> though some of them might require a client/server setup.</li> </ul> <p>For the code I worked with that was producing this issue, you can find it <a href="https://github.com/Swangeon/HCC/blob/main/select/select-implmentation.cpp">here</a>.</p> <h2 id="looking-forward">Looking Forward</h2> <p>Something that has been passed for now is the OpenVPN patchset I made so that the program can run on Haiku. Looking forward, I want to try and get my main commit <a href="https://review.haiku-os.org/c/haiku/+/6608">here</a> through before I continue to add more features to the driver to setup a good base for those who will work on this project after me. I am also working on in-depth documentation about how the TUN/TAP Driver and Interface work for those same people who will be here after me. Other than that, however, have a working build of the dynamic driver loading feature that I talked about last time as well that uses the <code>devfs_rescan_driver</code> function has been the life saver for this feature to work and I am just on the feature of uninitializing the driver entry&rsquo;s in devfs. Ideally, for the future, after the TUN/TAP driver base commit gets accepted then I will immediately put the dynamic driver loading feature commit up to then get that feature on the driver.</p> <p>Thank you all for reading all my posts over this summer and man, time goes by fast. I am so grateful that I got this opportunity and for you guys to be here along the journey with me. Will update for one last time about GSoC next week!</p>[GSoC 2023] .NET Developer Platform - Final Reporthttps://www.haiku-os.org/blog/trungnt2910/2023-08-20_gsoc_2023_dotnet_port_final_report/trung nguyenSun, 20 Aug 2023 00:00:00 +0000https://www.haiku-os.org/blog/trungnt2910/2023-08-20_gsoc_2023_dotnet_port_final_report/<h1 id="project-overview">Project overview</h1> <p>This project, a part of Google Summer of Code 2023, aims to port the .NET Developer Platform - a popular open-source framework - to Haiku, following various requests from the community to have a way to build C# or run .NET applications on this OS.</p> <p>The project picks up an incomplete port in 2022 by myself - <a href="https://github.com/trungnt2910">@trungnt2910</a> - and <a href="https://github.com/jessicah">@jessicah</a> and brings essential components of the .NET platform (its runtime and SDK) to Haiku. It also provides infrastructure to continuously build and distribute the Haiku version of .NET, and an additional workload allowing developers to build Haiku-specific applications.</p> <p>While working on the port, I also explained my implementation choices as well as provided an insight on how .NET interacts with the Haiku ecosystem through my five progress <a href="https://www.haiku-os.org/blog/trungnt2910/">blog posts</a>.</p> <p>It is possible to download and try .NET for Haiku by following the instructions at my <a href="https://github.com/trungnt2910/dotnet-builds">dotnet-builds</a> repository. The Haiku workload can be installed by following the steps at <a href="https://github.com/trungnt2910/dotnet-haiku">dotnet-haiku</a>. My code contributions span a wide range of commits and repos, and are listed <a href="#list-of-contributions-during-gsoc">below</a>.</p> <h2 id="completed-objectives">Completed objectives</h2> <p>I have completed most of the objectives mentioned in my original proposal:</p> <ul> <li>Rebased the existing .NET 7 port to the latest version of .NET 8.</li> <li>Ported the CoreCLR, system libraries, .NET SDK and <code>dotnet</code> CLI to Haiku.</li> <li>Tested .NET with some popular third-party libraries like <code>GtkSharp</code> and <code>FNA</code>.</li> <li>Generated .NET bindings for Haiku API kits, allowing developers to create Haiku-native apps with C#.</li> </ul> <h2 id="unresolved-issues">Unresolved issues</h2> <p>There are still a few problems I cannot solve as the GSoC period ends:</p> <ul> <li>Unmerged patches: Many patches to .NET have not been accepted. This might be because the GSoC coding period lies in the later stages in preparation for .NET 8 - a LTS release.</li> <li>Inability to build .NET from source on Haiku: Currently, building .NET requires certain binary dependencies which may not exist on Haiku yet. <code>clang</code> support on Haiku is also poor, causing problems when building native components.</li> <li>Lack of testing: Tests for native components run well on Haiku, but other tests require the ability to build the whole framework from source.</li> <li>Unimplemented features: Some less essential yet complicated portions are not implemented, most notably <code>System.Net.NetworkInformation</code>.</li> <li>Lack of HaikuPorts recipe: .NET 8 is still in prerelease, so it cannot be packaged on HaikuPorts yet.</li> </ul> <h2 id="future-work">Future work</h2> <p>These are the work I expect to do in the near future after GSoC ends:</p> <ul> <li>Continue working with .NET developers to upstream patches. There might be more chance for major changes like this to land on .NET 9, being a STS release.</li> <li>Provide .NET builds until at least the release of .NET 8.</li> <li>Watch for updates at <a href="https://github.com/dotnet/dotnet">dotnet/dotnet</a>. This is the official way to build .NET entirely from source. It currently only supports Linux, but when support grows to other platforms, this may be the solution for some issues faced when trying run a .NET source build on Haiku.</li> <li>Respond to any community requests through GitHub issues.</li> </ul> <p>If there is more community interest, I can also continue to polish the .NET Haiku workload by extending support to more kits, generating documentation, and adding more Haiku-specific functionality like compiling resources definitions or generating HaikuPorts recipe files.</p> <h2 id="experience-gained-during-gsoc">Experience gained during GSoC</h2> <ul> <li>Kernel and system software development: <ul> <li>Fixed 6 different kernel memory bugs.</li> <li>Implemented UNIX domain datagram sockets, an important missing feature.</li> <li>Made 8 other patches fixing problems and improving system software.</li> </ul> </li> <li>Debugging: Fix a variety of problems, including 3 bugs overlooked by Microsoft, using a creative combinations of tools and techniques, in environments lacking a good debugger.</li> <li>DevOps: Set up infrastructure surrounding the project, including 2 CI workflows, a NuGet feed, and a webhook connecting repositories together.</li> <li>Working with code without documentation: Analyzing undocumented code from both .NET and Haiku to solve problems as well as writing 5 blog posts (around 1500 lines) documenting the solutions.</li> <li>Communicating with different parties and stakeholders: Actively working with 3 different organizations to ensure functional software, and frequently engaging with the community during the project.</li> </ul> <h1 id="technical-details">Technical details</h1> <h2 id="project-structure-and-infrastructure">Project structure and infrastructure</h2> <p>This project mirrors some of the complexity of the official one from .NET. It spans across multiple repositories and makes extensive use of free infrastructure provided by GitHub.</p> <h3 id="forks-of-net-repositories">Forks of .NET repositories</h3> <p>Each repository of <code>dotnet/{repo_name}</code>, when ported to Haiku, is forked to <code>trungnt2910/dotnet-{repo_name}</code>. There are currently four of these repos:</p> <ul> <li><a href="https://github.com/trungnt2910/dotnet-runtime">trungnt2910/dotnet-runtime</a>: The core runtime.</li> <li><a href="https://github.com/trungnt2910/dotnet-sdk">trungnt2910/dotnet-sdk</a>: The SDK.</li> <li><a href="https://github.com/trungnt2910/dotnet-msbuild">trungnt2910/dotnet-msbuild</a>: MSBuild.</li> <li><a href="https://github.com/trungnt2910/dotnet-arcade">trungnt2910/dotnet-arcade</a>: Some build scripts synchronized across multiple .NET repositories.</li> </ul> <p>For each of these repos, the default branch <code>haiku-dotnet{latest_version}</code> is updated with the changes in the <code>main</code> branch upstream. Currently, this branch is <code>haiku-dotnet9</code>.</p> <p>Older versions like <code>haiku-dotnet8</code> are updated with .NET&rsquo;s <code>release/{version}</code> stable release branches.</p> <p>For repositories with more than one patch applied, such as <code>trungnt2910/dotnet-runtime</code>, each patch is kept in a separate branch, <code>dev/trungnt2910/{feature_name}</code>. Currently, the active branches are:</p> <ul> <li><code>dev/trungnt2910/haiku-config</code>: Tracking unmerged pull request <a href="https://github.com/dotnet/runtime/pull/86391">#86391</a>.</li> <li><code>dev/trungnt2910/haiku-pal</code>: Native (C++) support for the runtime on Haiku.</li> <li><code>dev/trungnt2910/haiku-lib</code>: Managed (C#) support for system libraries on Haiku.</li> </ul> <p>This separation of branches makes opening and keeping track of pull requests easier.</p> <h3 id="automatic-net-builds">Automatic .NET builds</h3> <p>The forks mentioned above are configured to trigger a webhook on push. Whenever the push is detected to be on any of the <code>haiku/dotnet{version}</code> branches, a CI run is triggered on <code>trungnt2910/dotnet-builds</code>. This produces a complete build of .NET for Haiku, containing the runtime and the SDK, cross-compiled from Linux.</p> <p>The same CI is also triggered when:</p> <ul> <li>Some script has been modified in the <code>trungnt2910/dotnet-builds</code> repository.</li> <li>A new week has started according to GMT time. These weekly builds helps detect any potential problems caused by updates on the Haiku side.</li> <li>A request from the owner of the <code>trungnt2910/dotnet-builds</code> repository has been received.</li> </ul> <p>When the CI run succeeds, artifacts are uploaded, NuGet packages are pushed, and a release containing a downloadable tarball is created. The custom <code>dotnet-install.sh</code> script included in this repository relies on these releases.</p> <h3 id="net-sdk-workload-for-haiku">.NET SDK workload for Haiku</h3> <p>The .NET SDK workload for Haiku (<code>trungnt2910/dotnet-haiku</code>) is built (nearly) from scratch by me and is not based on an existing repository by <code>dotnet</code>. It is built separately from the rest of the runtime.</p> <p>The CI for this repository only triggers whenever a new push is done to <code>master</code>. (This might be problematic when Haiku makes a header change, potentially breaking <code>CppSharp</code>, but suffices for now).</p> <h3 id="net-haiku-nuget-feed">.NET Haiku NuGet feed</h3> <p>Each GitHub account has a free NuGet feed associated with GitHub packages. Both <code>dotnet-builds</code> and <code>dotnet-haiku</code> push their resulting NuGet packages to my personal feed, <code>https://nuget.pkg.github.com/trungnt2910/index.json</code>. Many core functionality of .NET requires certain NuGet packages to be available. For Haiku, these packages are downloadable from the mentioned feed.</p> <p>I cannot upload the packages to the default public feed, <code>nuget.org</code>, because:</p> <ul> <li>Most packages published on NuGet are there forever. They cannot be removed. This can be undesirable as the packages I am working with are unstable and are for testing purposes only.</li> <li>Many packages' names start with <code>Microsoft.</code>, which is reserved for official Microsoft packages only. (If Haiku decides to make my C# bindings official, we could also reserve the <code>Haiku</code> prefix on NuGet when our packages are ready for release).</li> </ul> <h2 id="list-of-contributions-during-gsoc">List of contributions during GSoC</h2> <h3 id="haikuhaiku">haiku/haiku</h3> <h4 id="virtual-memory-management-improvements">Virtual memory management improvements</h4> <p>.NET makes complicated use of its virtual memory pages to store its JIT&rsquo;ed code and its heap. This reveals a lot of bugs as well as a few missing features on Haiku.</p> <ul> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6388">kernel/vm: Fix area_for with PROT_NONE address (#6388)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6389">kernel/vm: Allow more maximum protection for mmap&rsquo;ed areas (#6389)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6391">kernel/vm: Make cut_area respect overcommitting flag (#6391)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6392">kernel/vm: unlock cache before unmapping addresses (#6392)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6395">kernel/vm: handle page protections in cut_area (#6395)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6496">kernel/vm: check if page is in area (#6496)</a></p> </li> <li> <p><a href="https://review.haiku-os.org/c/haiku/+/6616">kernel/vm: Add clone_memory syscall (#6616)</a> (Unmerged)</p> </li> </ul> <h4 id="socket-improvements">Socket improvements</h4> <p>Haiku&rsquo;s implementation of UNIX sockets was incomplete, most notably <code>SOCK_DGRAM</code> datagram sockets. This prevented some .NET applications from functioning properly.</p> <ul> <li><a href="https://review.haiku-os.org/c/haiku/+/6617">unix: Implement datagram sockets (#6617)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6816">unix: Implement SO_RCVBUF (#6816)</a></li> </ul> <h4 id="terminal">Terminal</h4> <p>Some terminal features that will reduce the need of Haiku-specific compile-time workarounds.</p> <ul> <li><a href="https://review.haiku-os.org/c/haiku/+/6386">termios: New ioctl: TIOCOUTQ (#6386)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6387">tty: Implement exclusive mode (#6387)</a></li> </ul> <h4 id="miscellaneous">Miscellaneous</h4> <ul> <li><a href="https://review.haiku-os.org/c/haiku/+/6383">headers/bsd: Export pthread_attr_get_np</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6385">headers/posix: Add TIOCM_RNG as a synonym for TIOCM_RI (#6385)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6390">kernel/team: Allow retrieving more attributes (#6390)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6556">libroot: fix pthread_[g/s]etschedparam (#6556)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6716">headers/os: Make headers generator-friendly (#6716)</a></li> <li><a href="https://review.haiku-os.org/c/haiku/+/6718">headers: Explicitly hide BAlert functions (#6718)</a></li> </ul> <h3 id="haikubuildtools">haiku/buildtools</h3> <ul> <li><a href="https://review.haiku-os.org/c/buildtools/+/6384">gcc/config: Drop cdecl and stdcall built-in defines (#6384)</a></li> </ul> <h3 id="dotnetruntime">dotnet/runtime</h3> <p>The pull requests below adds support for Haiku and fixes some bugs overlooked by .NET engineers that only surface when running on this OS.</p> <ul> <li> <p><a href="https://github.com/dotnet/runtime/pull/86303">Initial build configuration for Haiku (#86303)</a></p> </li> <li> <p><a href="https://github.com/dotnet/runtime/pull/86207">[VM] Fix potential double free (#86207)</a></p> </li> <li> <p><a href="https://github.com/dotnet/runtime/pull/86843">[libs] Fix poll events conversion (#86843)</a></p> </li> <li> <p><a href="https://github.com/dotnet/runtime/pull/87119">[VM] Fix potential undefined behavior (#87119)</a></p> </li> <li> <p><a href="https://github.com/dotnet/runtime/pull/86391">Haiku: Configuration support (#86391)</a> (Unmerged)</p> </li> </ul> <p>Apart from the pull requests, the repository also contains a few commits, regularly rebased on top of the latest commits from upstream. At the time of writing, these commits are:</p> <ul> <li><a href="https://github.com/trungnt2910/dotnet-runtime/commit/ba3315a5">Haiku: Initial CoreCLR support (ba3315a5)</a></li> <li><a href="https://github.com/trungnt2910/dotnet-runtime/commit/62f9a0a8">Haiku: Initial managed libraries support (62f9a0a8)</a></li> </ul> <h3 id="dotnetarcade">dotnet/arcade</h3> <ul> <li> <p><a href="https://github.com/dotnet/arcade/pull/13437">Haiku: Fix infrastructure support (#13437)</a></p> </li> <li> <p><a href="https://github.com/dotnet/arcade/pull/13755">Haiku: Update arcade support (#13755)</a> (Unmerged)</p> </li> </ul> <h3 id="other-net-repositories">Other .NET repositories</h3> <p>As the pull requests in <code>dotnet/runtime</code> are still pending, it does not make sense to open patches to these higher-level repositories that depend on the runtime. The commits below are therefore kept in my personal forks.</p> <p>They are all small patches to make the corresponding components aware of Haiku.</p> <ul> <li><a href="https://github.com/trungnt2910/dotnet-sdk/commit/103b9510">dotnet/sdk: Add support for Haiku (103b9510)</a></li> <li><a href="https://github.com/trungnt2910/dotnet-msbuild/commit/55321f23">dotnet/msbuild: Add support for Haiku (55321f23)</a></li> </ul> <h3 id="monocppsharp">mono/CppSharp</h3> <p>These commits are related to bugs encountered when generating C# bindings for the Haiku API.</p> <ul> <li><a href="https://github.com/mono/CppSharp/pull/1741">CSharpExpressionPrinter: Wrap expression in parenthesis (#1741)</a></li> <li><a href="https://github.com/mono/CppSharp/pull/1745">CSharpExpressionPrinter: Recurse into operands (#1745)</a></li> <li><a href="https://github.com/mono/CppSharp/pull/1747">CSharp: More default parameter fixes (#1747)</a></li> <li><a href="https://github.com/mono/CppSharp/pull/1748">Array marshalling (#1748)</a></li> <li><a href="https://github.com/mono/CppSharp/pull/1752">SymbolResolver: Use filename when path cannot be found (#1752)</a></li> <li><a href="https://github.com/mono/CppSharp/pull/1753">CSharpSources: Dereference pointer variables (#1753)</a></li> </ul> <h3 id="trungnt2910dotnet-builds">trungnt2910/dotnet-builds</h3> <p>This repository contains CI scripts, installation scripts, and some documentation.</p> <p>The last commit done within GSoC 2023 is <a href="https://github.com/trungnt2910/dotnet-builds/tree/363e1db6">363e1db6</a>.</p> <h3 id="trungnt2910dotnet-haiku">trungnt2910/dotnet-haiku</h3> <p>This repository contains binding generators for the Haiku API, source code for the .NET workload for Haiku, CI scripts, installation scripts, and some documentation for this component.</p> <p>The last commit done within GSoC 2023 is <a href="https://github.com/trungnt2910/dotnet-haiku/tree/46876b14">46876b14</a>.</p> <h1 id="conclusion">Conclusion</h1> <p>Time really flies, and now my project finally has to end.</p> <p>While multiple factors have prevented an ideal outcome presented in the original proposal, I hope my work would still help the Haiku community gain access to more applications and develop native software in more innovative and modern ways.</p> <p>Google Summer of Code has surely gave me invaluable experience that cannot be replaced by any university courses. I look forward to participating this program for the coming years, and continue contributing to open source in general.</p> <h1 id="acknowledgements">Acknowledgements</h1> <p>Firstly, I would like to thank my mentors, nielx, and especially jessicah, for connecting with me before the project and providing me with various help.</p> <p>I would like to thank the Haiku organization, which has guided me to open source ever since I started coding in 2019, and its members, especially waddlesplash, pulkomandy, axeld, and korli, for spending time reviewing my thousands of lines of changes.</p> <p>I appreciate all the responses and help I received from members on Haiku&rsquo;s IRC channels, especially X512. I appreciate all the supportive comments and reactions from every Haiku community member. Special thanks to begasus who always follow every of my progress reports!</p> <p>I would also like to thank the members .NET Foundation, in particular <a href="https://github.com/am11">@am11</a>, for always supporting the Haiku port effort, despite not having direct responsibility in this GSoC project.</p> <p>Last but not least, I would like to thank my family for all the valuable support for my application and participation in this program.</p>[GSoC 2023] Improving Icon-O-Matic Final Reporthttps://www.haiku-os.org/blog/zardshard/2023-08-18_gsoc_2023_improving_icon-o-matic_final_report/zardshardFri, 18 Aug 2023 11:21:50 -0400https://www.haiku-os.org/blog/zardshard/2023-08-18_gsoc_2023_improving_icon-o-matic_final_report/<h2 id="what-is-icon-o-matic">What is Icon-O-Matic?</h2> <p><img src="../files/blog/zardshard/icon_o_matic.png" alt="Screenshot of Icon-O-Matic"></p> <p>There&rsquo;s a good chance that not everyone reading this article will know what Icon-O-Matic is, so I&rsquo;ll start by explaining what it is. Icon-O-Matic is a vector graphics editing program like Illustrator or Inkscape. It is specifically made to work with Haiku&rsquo;s custom HVIF vector graphics format. This format is similar to the SVG format, except optimized to be much, much smaller. The blog post <a href="http://blog.leahhanson.us/post/recursecenter2016/haiku_icons.html">&ldquo;500 Byte Images: The Haiku Vector Icon Format&rdquo;</a> provides a more in-depth discussion for those interested.</p> <p>If you want to try Icon-O-Matic yourself, you will need to first install the Haiku operating system. Afterwards, you can launch it from the applications menu.</p> <h2 id="my-goals">My goals</h2> <p>The goals of my project were rather broad: to add features, refactor code, and fix bugs. I had, therefore, a lot of leeway to do what interested me. My early plans were detailed in a blog post entitled <a href="https://www.haiku-os.org/blog/zardshard/2023-05-05_gsoc_2023_plans_for_improving_icon-o-matic/">&quot;[GSoC 2023] Plans for improving Icon-O-Matic&quot;</a>. They changed rather significantly over the course of the project.</p> <h2 id="what-i-got-done">What I got done</h2> <p>Below is a list of the things I got done during GSOC. For those interested in seeing the full list, it can be seen <a href="https://review.haiku-os.org/q/owner:0azrune6%2540zard.anonaddy.com+mergedbefore:2023-08-28">here</a>. This list does not include commits made outside of GSOC.</p> <h3 id="fixed-memory-leaks">Fixed memory leaks</h3> <p>For those not familiar with it, a memory leak occurs when a program requests memory from the operating system and later forgets to tell the operating system that it&rsquo;s done with it. This leads to memory being wasted. In the worse-case scenario, the program eventually uses up all of the system&rsquo;s memory.</p> <p>One of my GSoC project ideas was to make a program to detect memory leaks. I had decided not to go with that. Later, I discovered that Haiku had a memory leak detector built-in! I wrote <a href="https://www.haiku-os.org/blog/zardshard/2023-05-23_how_to_find_memory_leaks/">a blog post</a> detailing how to use it so that others knew of its existence and how to use it. I was excited to use this memory leak detector on Icon-O-Matic, so, that was one of the first things I did.</p> <p>And, indeed, I did find and fix many memory leaks. Some of these leaks were caused by system libraries. This means that some of the bugfixes helped every application leak less memory.</p> <p><strong>Commits:</strong></p> <ul> <li><a href="https://git.haiku-os.org/haiku/commit/?id=943b5775f875e9f3f6bb8edc222289e02faa0ef9">Icon-O-Matic: Fix memory leak</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=0e86ca77e77224b270dc3dff905907940cc74ae6">Tracker: Fix memory leak</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=338fedd65a5bc57f3ec35b9c0d48ab5d8b013bd3">Tracker: Fix memory leak</a></li> </ul> <h3 id="added-reference-images">Added reference images</h3> <p><img src="../files/blog/zardshard/reference_images.png" alt="Screenshot of three reference images"></p> <p>This is probably the most useful feature that I added. This allows you to create an image in another program such as Inkscape or Illustrator and then use Icon-O-Matic to trace it. This is useful since Icon-O-Matic is required to make HVIF files but doesn&rsquo;t have as many features as many other programs. This also gives Icon-O-Matic greater ability to specialize in making small HVIF files as opposed to being a general-purpose vector graphics program.</p> <p><strong>Commits:</strong></p> <ul> <li><a href="https://git.haiku-os.org/haiku/commit/?id=098eaec6305ae804d3eb6c8e6c6aad790fb4cfb1">Icon-O-Matic: Add reference images</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=4ed150bdc36d063144a3552305c5d77ecac88427">Icon-O-Matic: Save alpha of reference images</a> (bugfix)</li> </ul> <h3 id="added-perspective-transformations">Added perspective transformations</h3> <p><img src="../files/blog/zardshard/perspective_transformer.png" alt="Screenshot of a grid being put into perspective"></p> <p>I believe this feature will be a lot less useful than reference images. Ironically, it also took quite a bit more time to implement than reference images. Implementing this feature also took the most math. I used quite a bit of linear algebra to figure out how to determine whether a perspective transformation was valid or not!</p> <p><strong>Commits:</strong></p> <ul> <li><a href="https://review.haiku-os.org/c/haiku/+/6808">agg: Pull in updated perspective transformation</a> (unmerged as of writing)</li> <li><a href="https://review.haiku-os.org/c/haiku/+/6801">Icon-O-Matic: Add perspective transformations</a> (unmerged as of writing)</li> <li><a href="https://sourceforge.net/p/agg/patches/6/">Fix multiple definitions linker error</a> (upstreams some changes to the AGG library)</li> </ul> <h3 id="reduced-repeated-code">Reduced repeated code</h3> <p>This shaved off 1,300 lines of code from Icon-O-Matic. This one is the only change that caused a regression, at least, as far as I&rsquo;m aware ;)</p> <p><strong>Commits:</strong></p> <ul> <li><a href="https://git.haiku-os.org/haiku/commit/?id=6427935280aaac0a1a4557bae55184708819560d">Icon-O-Matic: Generalize some classes</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=c5abd6a796982af7cc456f832086da2e4a13ad83">libicon: Notify Shape on transformer addition/removal</a> (regression fix)</li> </ul> <h3 id="miscellaneous-improvements">Miscellaneous improvements</h3> <p>There are various miscellaneous improvements that I made to Icon-O-Matic along the way. Some are bugfixes. Others introduce some simple features. None of these took very much work.</p> <p><strong>Commits:</strong></p> <ul> <li><a href="https://git.haiku-os.org/haiku/commit/?id=8ffe04780def2570c3806e6c18fa14a79fd105eb">Icon-O-Matic: Remove unused function parameter</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=16edf24a3c77400a81110adb8933f640bf210089">Icon-O-Matic: Remove unused dependency</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=a75a222b35d17cd83bc75253f3cd8e24a6a911f4">Icon-O-Matic: Remove dead homebrew translation system</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=c70aca1135bb1e1999db080457a40e9d36989582">Icon-O-Matic: Make left sidebar lists resizable</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=6da29a769f1e2afb5d98f0470b35497d7f3ef3ea">Icon-O-Matic: Fix window resizing bug</a></li> <li><a href="https://git.haiku-os.org/haiku/commit/?id=405c9dc3f27cf660facae5253303ea24aaaf382e">Icon-O-Matic: Improve saving-related error messages</a></li> <li>and various others</li> </ul> <p>A full list can be seen <a href="https://review.haiku-os.org/q/owner:0azrune6%2540zard.anonaddy.com+mergedbefore:2023-08-28">here</a>.</p> <h2 id="conclusion">Conclusion</h2> <p>It&rsquo;s surprising how long some of these things took to do. My original project proposal for GSoC included time estimates. Unsurprisingly, my plans changed over the course of GSoC. By the end of GSoC, I had only done one of the things I had originally planned to do: adding reference images. I had estimated that adding them would take 1 week. I was aware that things tended to take longer than estimated so I made room in my estimates for that. Despite that, adding reference images actually took, depending on how you count, 2-3 weeks!</p> <p>I also learned about the importance of communities surrounding open-source projects. These communities influence what happens on the project, give developers ideas and feedback, and find bugs. The developers can help other developers, give them tips, or, just as importantly, criticize their ideas. These communities are also a nice place for people to simply hang out and socialize.</p> <p>Thanks to Google for hosting Google Summer of Code. Without you, I would not be here. Thanks also to my mentors, PulkoMandy and Humdinger, and to the broader community surrounding Haiku. You have given me advice, pointed out pitfalls, and given me many ideas.</p>[GSoC 2023] VPN Support Project Update #6https://www.haiku-os.org/blog/pairisto/2023-08-17_gsoc_2023_vpn_support_project_update_6/pairistoThu, 17 Aug 2023 21:53:21 -0500https://www.haiku-os.org/blog/pairisto/2023-08-17_gsoc_2023_vpn_support_project_update_6/<h2 id="where-we-last-left-off">Where We Last Left Off</h2> <p>So last time I posted I was able to say that I got the client side for OpenVPN on Haiku working but not the server but I am proud to say that now both the server and client work extremely well now on Haiku :) I was able to get the server from not working to working and was able to get the latency for the entire VPN operation down from 1000ms average to anywhere between 2ms to 9ms (that&rsquo;s a caveat as that is without blocking which will be discussed later). I had a check in my tun_read() function wrong where I wanted any side that is non-blocking to send a signal to the other side&rsquo;s condition variable that something is now in their queue but had it backwards with a <code>not</code> statement in there. The simplest mistakes slip the mind huh :p Anyway, the server also follows suit with the faster latency so now that the read and write functions are basically complete, lets get on with what I was dealing with for the past 2ish weeks.</p> <h2 id="new-week-new-problems">New Week, New Problems</h2> <p>After I had gotten the read and write functions sorted out I wanted to move on to making sure the tun_close() function works properly as before it would just hang when <code>ifconfig down/delete</code> were called on the interface. I realized that the hanging behavior was due to the interface waiting for the reader thread to return something in the <a href="https://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/network/stack/device_interfaces.cpp#n575">down_device_interface()</a> function in device_interfaces.cpp. Well it sounds like a simple fix, notify the condition variable in tun_close() and job well done right? WRONG! Whenever I would add the NotifyAll() method into tun_close(), I would get a kernel panic saying:</p> <pre tabindex="0"><code>attempt to acquire lock (some random address here) twice on non-SMP system (last caller: 0x0000000000000000, value deadbeef) </code></pre><p>After the error was shown in KDL, it would show me that this was coming from the tun_read() <code>Wait</code> that I was doing on the condition variable where it specifically locks again. If you don&rsquo;t know how condition variable waiting works, here is a brief run down and look at the code <a href="https://cgit.haiku-os.org/haiku/tree/src/system/kernel/condition_variable.cpp#n302">here</a>:</p> <ol> <li>You pass in a locked mutex and depending on what language you are using, either the condition variable itself or Wait will be a method to a condition variable class like it is with Haiku</li> <li>It will then unlock the mutex</li> <li>Wait using a thread blocker and spinlock</li> <li>When notified, it will then exit the wait function and relock the mutex The last part was where the problem was. This was an extremely odd error to get since tun_write() also needs to notify the locked read thread but that has never done something like this. This error basically came about due to me trying to acquire the mutex at the same time so I decided to make the tun_close() function pause for a bit after it notified the mutex and started working again.</li> </ol> <p>Now one last thing that I mentioned in my last post was that OpenVPN takes about 99% CPU usage on Haiku which is still in a weird state of affairs for now. I had thought that since OpenVPN wants the driver in a non-blocking state to operate efficiently, we would need Asynchronous I/O to get the write and read calls to not block each other but we do not have this functionality (yet ;)). That was when Waddlesplash and x512 helped me with looking at the source code for OpenVPN and realized that it had a select/poll way of doing things instead of AIO. Now I have the select functionality working, however, this is where there are some problems as it works but not well as the average latency is above 2000ms and when using ping it drops more than 60% of packets on average. Hopefully I can fine tune things though I don&rsquo;t know how much more I can give what select itself is and how the implementation of select you have to do in Haiku is done but I will update you all on this.  </p> <h2 id="moving-forward">Moving Forward</h2> <p>Now that tun_close() is done, there is really only one thing that I want to finish up before I feel like the code is ready for a push to the main code base being that I want the driver to be dynamically allocated. The behavior that I am looking for is that when <code>ifconfig tap1 ... up</code> is called, I want it to be able to make a new entry in devfs with the name provided to ifconfig like in Linux. Currently, the way I have it set up is to have the entry in devfs made when Haiku starts which I personally don&rsquo;t want to push out like that since it&rsquo;s just a devfs entry that may never be used by the user. There are two immediate problems that I see being:</p> <ol> <li>The creation of a new devfs entry upon <code>open()</code>.</li> <li>How I will differentiate the application side from the interface side. The first problem will probably be solved by understanding how making an entry in devfs works though that will be learning so I will learn more about that. Maybe something like <a href="https://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/drivers/network/ether/etherpci/etherpci.c#n1408">this</a> could help? Moving onto the second problem, my current solution for it is creating another entry called <code>tun_interface</code> within <code>/dev/misc</code> that the interface calls on which helps with differentiating the two sides. However, I would rather have the interface just call on the name it was given to through <code>ifconfig</code> which would then bring up that devfs entry. So an <code>IOCTL</code> call when the interface is going up can probably do the trick if I just have the default <code>tun_struct</code> be in the app&rsquo;s form and then change it to the interfaces form through said call.</li> </ol> <p>Thank you all for reading, please feel free to leave a comment!</p>