Post

P4 Compiler Documentation

Posted
By P4 lang
12 min read
P4 Compiler Documentation

P4 Compiler Documentation

Mentors: Davide Scano, Fabian Ruffy
Contributor: Adarsh Rawat

Gsoc2024 - The P4 Language Consortium

Table of Contents

Project Overview

The primary objective of this project was to improve the documentation infrastructure of the P4 Compiler (p4c). This involved upgrading tools, automating deployment processes, enhancing the user interface, and ensuring that the documentation is both comprehensive and accessible. The project was aimed at making the documentation easier to maintain and more useful for both developers and users of the P4 language.

image

Technical Considerations

[!NOTE]
Previews in this report are mockups and do not reflect the current state of the documentation. They were used for iterative development during the project.

Doxygen vs. Sphinx: Documentation Tools Comparison

Initially, I considered using Sphinx with Doxygen and Breathe for documentation [1][2]. However, after discussing with mentors and evaluating the tools, we decided to use Doxygen with Doxygen Awesome CSS for the following reasons:

  • Doxygen: Provides straightforward and easy-to-generate documentation but lacks mobile responsiveness. Examples of Doxygen-generated docs can be found here:
  • Doxygen + Awesome CSS: Addresses the mobile responsiveness issue and improves visual appeal with enhanced CSS styling. It allows for customization without modifying the HTML structure and includes dark mode support. You can view an example here:
  • Doxygen + Breathe / Exhale + Sphinx: Sphinx offers greater flexibility and control over documentation structure compared to Doxygen’s markup. It’s particularly useful for including tutorials. However, we decided against it for our project to avoid added complexity, especially since tutorials would be maintained separately.

After discussions with mentors, it became clear that Doxygen with Doxygen Awesome CSS met all our needs while avoiding the additional complexity of incorporating Sphinx and Exhale. This choice allowed us to utilize a tool familiar to maintainers and contributors without introducing a new layer of complexity.

Documentation Development and Build Process

During the early stages of development, I utilized a personal repository with Netlify to create and share website mockups. This approach allowed for iterative testing and refinement before deploying to the official GitHub Pages site. The following mockups were created during this phase:

Deployment Options : Transition to GitHub Pages

After finalizing the mockups and build process, the documentation was deployed on GitHub Pages. This transition from Netlify to GitHub Pages streamlined hosting, ensured consistent integration with the main repository, and eliminated additional costs and administrative overhead. The choice was made after evaluating platforms like Netlify and Vercel and consulting with mentors.

Workflow Automation

Doxygen Automated Builds Workflow

PR - Configure: DOXYGEN for documentation generation
PR - Docs : Configure Doxygen GitHub pages deploy Action
An automated workflow was set up to handle Doxygen documentation builds. This process ensures that the documentation is updated consistently with the latest changes, providing a reliable and up-to-date reference for all contributors.

Automated PR Preview Workflow

PR - Feat: PR Preview Workflow via GitHub Pages To make reviewing documentation changes smoother and more transparent, I introduced a new PR preview workflow. This update enhances how we assess documentation updates before they merge into the main branch by using GitHub Pages for live previews. This means we can see exactly how changes will look in real-time, making it easier to review and refine documentation. Preview of PR

Key Updates

  • PR Preview: Automatically builds and deploys each documentation PR to a unique GitHub Pages URL.
  • Comment Updates: Adds/Updates comments with preview links and build status.
  • Broken Builds: Detects and reports any build issues.

    Benefits

  • Immediate Feedback: Real-time previews for faster review and adjustments.
  • Automatic Notifications: Alerts for build statuses and failures.

    Usage

    When a PR is tagged with documentation, the workflow deploys a preview to GitHub Pages and posts a comment with the preview link and build status.

Background : Challenges with PRs from External Forks

Handling pull requests (PRs) from external forks presents unique challenges in GitHub Actions, primarily due to security restrictions imposed by GitHub. When a PR is triggered from a fork, the GITHUB_TOKEN generated for that workflow run has limited permissions, specifically lacking write access to the repository. This limitation is designed to prevent unauthorized actions that could potentially compromise the security of the repository. As a result, actions such as commenting on a PR, which require write access, fails with an error message like Resource not accessible by integration.

Workaround

Using pull_request_target event instead of pull_request: This event runs in the context of the base repository, where the workflow file is stored, and grants the GITHUB_TOKEN the necessary write permissions. However, this approach comes with its own set of risks, as it executes code from the forked repository with elevated privileges, which could potentially be malicious.

Background : Label Check Implementation for pull_request_target Trigger

In my GitHub Actions workflow, I encountered a limitation with the pull_request_target trigger. Specifically, the conditional check using contains on PR labels wasn’t functioning as expected.

1
2

# This condition didn't work with the pull_request_target trigger
if: ${{ contains(github.event.pull_request.labels.*.name, 'documentation') }}

Workaround

Custom Label Check To ensure that only pull requests with the ‘documentation’ label trigger the workflow, I implemented a custom label check using actions/github-script@v7(GitHub’s REST API).

This script iterates over the labels attached to the PR and returns true if the documentation label is present. This condition is then used to control subsequent steps in the workflow.

1
2
3
4
5
6
7
8
9
10
11
12
13

- name: Check PR Label
  id: check-label
  if: ${{ github.event_name == 'pull_request_target' }}
  uses: actions/github-script@v7
  with:
    script: |
      const { data: labels } = await github.rest.issues.listLabelsOnIssue({
        owner: context.repo.owner,
        repo: context.repo.repo,
        issue_number: context.issue.number
      });
      const hasDocumentationLabel = labels.some(label => label.name === 'documentation');
      return hasDocumentationLabel;

Doxygen v1.12.0 Update: Enhancements and Open Source Impact

PR - Bump Documentation build workflow action from Doxygen v1.11.0 to v1.12.0
Background on Doxygen v1.12.0 Feature Doxygen v1.12.0 introduced a critical feature enabling the use of GitHub-flavored comments, allowing Doxygen commands to be hidden in GitHub previews using the <!--! ... --> syntax. This feature was the direct result of a feature request [3] I submitted to address the issue of visible Doxygen commands cluttering the rendered views of Markdown documents(README Files) on GitHub.

Impact of the Update

  • Enhanced Documentation: By adopting this new comment style, all Doxygen commands are now hidden in GitHub previews, making the documentation cleaner and more professional.
    PR - Docs : Content Organization and Documentation Updates

  • Updated Workflow: I updated the Doxygen workflow action from v1.11.0 to v1.12.0, ensuring that all documentation now takes advantage of this new feature. This update mandates the strict use of Doxygen v1.12.0 for consistent output across the project.
  • Dependency Management: Dependencies were updated accordingly to support this new Doxygen version. This change requires all contributors to upgrade to v1.12.0 to avoid confusion during documentation merges, particularly when using commands like <!--!\include{doc} "../lib/README.md"--> as they will not be processed by previous versions of doxygen.

Key Improvements and Achievements

Enhanced Documentation Quality

Improved Documentation Accessibility

  • Documentation Sync and Cleanup: Updated and fixed broken links to align with the latest codebase.
  • Centralized Hosting: Deployed documentation to GitHub Pages for real-time updates and accessibility.
  • Permanent Links: Added links to specific sections for easier navigation and sharing.

    Enhanced Visual and Functional Features

  • Dark Mode and Formatting: Introduced dark mode and improved navigation bar formatting for a better reading experience.
  • Customized Theme: Adopted a P4 color scheme for a consistent visual experience.

Efficient Onboarding and Project Management

Supporting Pull Requests

Contributions to P4 Tutorials

Contributions to P4 Compiler

Closing Note

A Touch of Fun in the P4 Compiler Architecture Diagram

PR - Docs : Add Home page for documentation
In the spirit of keeping things lively, we came up with the idea of sneaking a little Easter egg into the P4 Compiler’s documentation during one of our team meetings. If you’re feeling curious (or just bored), try clicking on the P4 Compiler logo in the top left corner of the homepage. What happens next? Well, the architecture diagram decides to throw a tiny celebration by animating its lines, as if to say, “Surprise! You found us!”

Interactive Animation Toggle in P4 Compiler Architecture Diagram

This quirky touch isn’t just for giggles — it’s a subtle reminder that even in the serious world of compiler documentation, a bit of fun can go a long way. Plus, it might just keep you on your toes, wondering what other surprises might be hiding in the corners of the site. Spoiler: there aren’t any… yet.

Reflections and Learnings

During this project, I often had to change my approach and come up with new ideas to solve problems. I learned how a bit of help from others can quickly solve issues that might take me hours to figure out on my own.

I learned the importance of learning in public. Initially, I felt skeptical about sharing my progress and failures openly, but the overwhelmingly positive response and genuine support from the community and mentors reassured me. This encouragement was instrumental in helping me work confidently towards the project.

Reflections on Open Source Collaboration

This experience highlighted the immense power of open-source tools and communities. The rapid incorporation of the feature request into Doxygen’s latest release underscored the collaborative spirit and responsiveness that are the hallmarks of open source. It also reinforced the importance of contributing to open-source projects, as even small changes can have a significant impact on a wide range of users.

In this project, I saw firsthand how contributions from the community can lead to enhancements that benefit everyone. Working within an open-source ecosystem allowed me to both give back and learn from others, reinforcing the idea that open-source software isn’t just about code — it’s about community.

What’s Left to Do (Future Steps)

Documentation is an ever-evolving aspect of the project. Future work includes regularly updating the documentation to reflect new features, changes, and improvements.

The PR - Docs: Update Contributor Documentation Guidelines provides detailed instructions for future contributors on how to use Doxygen and maintain the documentation effectively. These guidelines will play a crucial role in ensuring that contributors can consistently update and expand the documentation. Continued attention to these instructions, along with regular updates and automated changelog integration, will help keep the documentation accurate and comprehensive.

Final Thoughts

This was one of the best experiences I’ve ever had. I sincerely thank Google and my project mentors for selecting me for the GSoC program and giving me the opportunity to discover how amazing coding can be.

References

[1]: Medium article on using Sphinx, Doxygen, and Breathe: C++ Documentation with Doxygen, CMake, Sphinx & Breathe
[2] Microsoft Dev Blog on functional C++ documentation: Clear Functional C++ Documentation with Sphinx, Breathe, and Doxygen
[3]: Feature Request : Capability to render GitHub flavor Markdown comments
GSOC 2024
gsoc2024 documentation
This post is licensed under CC BY 4.0 by the author.