Section</th>	Data stored</th>	Type of data</th></tr></thead>
Header</td>	size of data (size)</td>	`unsigned long</code></td></tr>`
Header</td>	size of provided data (real size)</td>	`unsigned long</code></td></tr>`
Header</td>	pointer to next chunk (next)</td>	`void *</code></td></tr>`
Header</td>	pointer to previous chunk (prev)</td>	`void *</code></td></tr>`
data</td>	provided data (could be any size)</td>	N/R</td></tr>
data</td>	tail (could be any size)</td>	N/R</td></tr> </tbody></table> The size of the provided data is only here for debug information and `show_alloc_mem</code>.</p>` Pros and cons</h3> The pros of overhead instead of exported bookkeeping are that it is faster to retrieve data related to a chunk with a pointer (in free</code> or realloc</code>, for example) since it only requires subtracting the header size from the pointer to retrieve its header.</p> It also takes less space since it does not contain pointers to data.</p> It also requires fewer calls to mmap</code> (which are fairly slow) since it is not allocated elsewhere; data and header are allocated in a single call.</p> The cons are that a non-cautious user writing out of provided data bounds might ruin the bookkeeping data and ruin malloc… But well, nobody is supposed to write out of bounds, right?</p> Bins</h2> Similarly to a chunk, bins have a preceding header. The only differences are that the size corresponds to the full zone size, header included, that the real size field is unused, and that pointers point to bins instead of chunks.</p> malloc</h2> When calling malloc</code>, if the size is not too high for a bin (then it just performs a fitted allocation), it will iterate in the proper bin (corresponding to the requested size) and find a free chunk with the proper size. This helps to prevent bin fragmentation. If it reaches the end of a bin and there is enough space left in the bin, it will attribute a chunk in the bin. If there is not enough space left in the bin, it will create a new bin.</p> If several contiguous freed chunks’ total size matches the new chunk’s size, they will be joined as a single chunk, again, to prevent fragmentation.</p> free</h2> When calling free</code>, if the size is not too high for a bin (then it will just unmap the chunk and sort out next/prev pointers), it will check if the provided chunk is at the end of a bin. If so, the chunk will be de-attributed, then it will iterate backward to find contiguous chunks to de-attribute them. This helps to prevent bin fragmentation.</p> If the bin ends up being empty, it is de-allocated to reduce the program’s memory footprint.</p> realloc</h2> When calling realloc</code>, if the size is not too high for a bin (which will be covered in a few lines), it will run a few checks:</p> If the new size equals the current real size, it will just return the provided pointer.</p> </li> If the new size is less than the current real size, it will adjust the chunk’s real size and return the provided pointer.</p> </li> If it is greater than the current real size but less than the chunk’s data size, it will adjust the chunk’s real size and return the provided pointer.</p> </li> </ul> In the two previous cases, if the chunk is at the end of a bin, the chunk will be expanded (if there is enough space left) or trimmed to fit the new size.</p> If the new size is greater than the chunk’s data size (or bin’s left size if it is at the end of a bin), it will store the chunk’s data in a buffer, free the chunk, call malloc</code> to get a new chunk, and copy the buffer into the new chunk’s data.</li> </ul> Similarly, if the size is too big for a bin, it will try to expand or trim the chunk with mmap</code>. If it works, the chunk’s size and real size are changed accordingly and the provided pointer is returned. If it does not work, it will copy the chunk’s data to the new chunk and return a pointer to the new chunk’s data.</p> Bonus part code explanation</h1> None of the bonuses really require an explanation, besides eventually de-fragmentation…</p> Preventing fragmentation is already covered in the mandatory part (to be fair, if one wants to do this bonus, it is recommended to code the mandatory part with de-fragmentation in mind…).</p> Making the lib thread-safe is not that hard; the easiest implementation is to lock/unlock a mutex before each call of the exposed functions. It could eventually be optimized, but for little to no gain at my scale.</p> To make it fancier, I added macros to exclude thread-safe logic at build time when not required.</p> I also added some debug information on show_alloc_mem</code> when building with a specific flag, it prints a dump of the chunks, along with more details about the sizes of the chunks and bins.</p> Overview of my infrastructure Millefeuille — Thu, 18 Jul 2024 00:00:00 +0000 Introduction</h2> </i>Important</p> This blog post presents my current infrastructure. An updated version will be published when major modifications are made. This article will then be categorized as obsolete.</p> </blockquote> In this article, I’m going to present the infrastructure I’ve set up to self-host my services. This overview will give you an idea of the hardware and software I use, as well as the reasons that led me to make this choice.</p> Why I chose self-hosting</h2> I chose to host my services for several reasons:</p> Total control</strong>: To be able to configure and adjust every aspect of my infrastructure to my needs without depending on an external provider.</li> Confidentiality</strong>: My data belongs to me, and self-hosting ensures that there are fewer overly interested parties spying on my data.</li> Learning</strong>: Setting up and managing this infrastructure has helped me a lot acquiring my current knowledge.</li> For fun</strong>: I like experimenting with technology and finding solutions to technical problems. It’s a passion that occupies my free time in a productive way.</li> </ul> Constraints</h2> I don’t have a lot of money to put into electricity bills, so it was and is important that the hardware I chose is not too energy-intensive. Additionally, low-energy equipment generally produces less heat, and I don’t like it when it’s hot in my apartment.</p> My infrastructure</h2> Hardware</h3> A Raspberry Pi 4 8 GB RAM</li> 16 GB SD card for the system</li> 128 GB USB SSD for the /var</code> folder</li> 3 TB HDDs for volume storage</li> </ul> </li> </ul> </i>Note</p> I’m using an SSD for the /var</code> folder to preserve SD cards and eMMC, while maintaining sufficient speed.</p> </blockquote> Due to budget constraints, which will soon be resolved, I’m only using a Raspberry Pi 4 and have had to remove certain elements such as monitoring.</p> Software</h3> I only use Docker and a few system tools on my Raspberry Pi. But here’s my “Must-Have” list:</p> vim / nano</li> htop / btop / top</li> git</li> curl / wget</li> ping</li> ip</li> dig</li> nslookup</li> </ul> I dockerize many services, the main ones being:</p> Traefik</strong>: for reverse proxy routing to my services</li> Authentik</strong>: for SSO on compatible services and forward proxy protection of services that don’t propose login</li> Portainer</strong>: to manage my containers</li> Gitea</strong>: for a “home” copy of my repositories. I also use it to store my private repositories for my infrastructure configuration (config files, compose files, etc.).</li> </ul> As well as all the other services that aren’t useful for infrastructure management, which I’ll detail in other articles.</p> Security</h2> Access</h3> I only expose the ports required for the services I want to make accessible, i.e., 80 and 443 for HTTP and HTTPS. If a service requires me to expose another port (such as my IRC server), I expose it if I want to access it from the outside.</p> SSH access to the machine is via the local network or my VPN.</p> SSL</h3> To have HTTPS on my services, I use a wildcard certificate as well as a standard certificate (*.millefeuille42.fr</code> and millefeuille42.fr</code>). My registrar (OVH) lets me obtain and renew them via their API, all free of charge.</p> If I can’t (Namecheap and Scaleway), I use the Let’s Encrypt resolver from Traefik to obtain and renew them automatically and dynamically.</p> Forward Proxy</h3> To protect my services that aren’t intended to be public (and/or that don’t provide authentication on their own, like Homer), I use Authentik’s Forward Proxy mode with Traefik. This allows me to manage access to these services from a single location.</p> Deployment of key services</h2> To lighten this article, I’ll detail the deployment of key tools (Traefik, Authentik, Portainer, Gitea) in another article. I’ll put the link here when it’s done.</p> Managing other infrastructures</h2> The services presented here permit me to manage other servers, particularly for close friends and family. For Portainer, all I need to do is to add an agent to the server, and Authentik provides branding according to the domain used for access. All that is left to do is to install and configure Traefik on the desired entry point. Authentik’s user management lets me isolate users of the different servers.</p> Conclusion</h2> Self-hosting is a rewarding adventure that permits you to regain control over your data and acquire many technical skills. Although it takes time and effort, the benefits in terms of privacy, control, and personal satisfaction are well worth it. If you’re passionate about technology and like a challenge, I strongly encourage you to give self-hosting a try. You won’t regret it.</p>

How I recovered from a wrong lvremove

Millefeuille — Thu, 23 Apr 2026 00:00:00 +0000

Have you ever hit Enter</code> on your keyboard and immediately felt the greatest anguish of your life?</p>

It was late in the afternoon, around 17:00. My team and I were doing some routine maintenance on a production environment. We needed to clean up a corrupt VHD, it was supposed to be a standard short five minute job… Instead, it turned into the most stressful hours of my life during which we struggled to recover an entire Storage Repository (SR) after I accidentally deleted all Logical Volumes (LV) in a Volume Group (VG).</p>


	
		</i>Note</p>
	
In the aforementioned production environment, each VM disk (in the VHD format) is stored as a LV and is exposed to the VM as a block device. A SR is the meta-name for a group of VHD managed by a specific driver, here LVM. One SR is represented by a single VG.</p>

</blockquote>
Here’s the story of how a simple syntax error caused a massive outage and how reading the man (and some bash scripts) saved the day by transforming a data loss into a 2 hours service interruption without a single byte of data lost.</p>
The Mistake</h2>
The job was simple: remove a single, corrupted Logical Volume.</p>
I executed the command: lvremove VG_NAME LV_NAME</code>. Any reader that once manipulated LVM could spot where the error lies… Usually a lvremove</code> command is written in one of these ways:</p>
lvremove</span></span> VG_NAME/LV_NAME</span>
</span>lvremove</span></span> /dev/VG_NAME/LV_NAME</span>
</span></code></pre>
Because I forgot a slash / didn’t specify the whole path, the command’s scope was evaluated much broader than intended, essentially targeting all LVs within that VG.</p>
To make matters worse, I’ve been asked asked for confirmation to delete the management (MGT) LV. I hit n</code> (no) to abort that specific deletion, but instead of exiting, LVM happily proceeded to delete all the other</em> logical volumes in the group without further confirmation.</p>
As I witnessed the terminal print line after line Logical volume "..." successfully removed</code>, my colleagues and I immediately realised what had happened. Production VMs suddenly lost their underlying disks. All VM I/Os halted.</p>
The Panic (and the missing backups)</h2>
After a small and contained panic attack, I managed to get control over myself pretty quickly considering what was at stake and that I couldn’t let my team take the stress for me.</p>
My first thought was,</p>

“No problem, LVM automatically backs up metadata.”</p>
</blockquote>
Normally, when LVM configuration changes, it saves a backup in /etc/lvm/archive</code>. You can just run a vgcfgrestore</code> and be back in business in minutes.</p>
We checked the archive directory. It was empty.</strong> Due to an edge-case in how this specific environment was configured, the standard LVM filesystem archives weren’t there. We needed to find another way.</p>
The Solution: The LVM Metadata Ring Buffer</h2>
Since we couldn’t rely on the standard LVM backups, I decided to look at the raw block device. Upon reading the man (yes, the man) I learnt that LVM maintains a circular ring buffer of recent configuration changes in the first ~1MB of the Physical Volume (PV) header. Allowing to essentially revert to any state immediately preceding an issue, let’s say a catastrophic typo.</p>

	
		</i>Note</p>
	
I wasn’t left handling this alone. In the meantime my colleagues were searching for alternative solutions.</p>

</blockquote>
Here is the step-by-step process I used to recover the volumes.</p>
1. Identify the Physical Volume</h3>
First, I needed the exact disk backing the PV for the lost VG.</p>
#</span></span> Identify the device ID backing the SR</span>
</span></span>pvs</span></span> |</span> grep</span></span> VG_NAME</span>
</span>#</span></span> Result: /dev/disk/by-id/DISK_SERIAL</span>
</span></span></code></pre>
2. Extracting the Metadata Records</h3>
Instead of blindly dumping the entire 1MB header with dd</code>, I used pvck -v</code> to pinpoint the exact offset and size of every metadata record stored in the ring buffer.</p>

	
		</i>Note</p>
	
I got really lucky here, depending on the version of LVM and some factors, it is possible that pvck</code> doesn’t find the metadata entries!</p>

</blockquote>
I made this little bash script to extract each historical state into its own raw file, and simultaneously parsed out the VHDs present in that state into a text file for easy comparison.</p>
pvck</span></span> -</span>v</span> /dev/disk/by-id/DISK_SERIAL 2</span>>&</span>1</span></span> |</span> \
</span></span>grep</span></span> '</span>Found LVM2 metadata record'</span></span></span> |</span> \
</span></span>sed</span></span> '</span>s/.*offset=\([0-9]*\), size=\([0-9]*\).*/\1 \2/'</span></span></span>|</span> \
</span></span>while</span> read</span></span> off sz</span>;</span> do</span>
</span>    echo</span></span> "</span>Offset: $</span>off</span></span> | Size: $</span>sz</span></span> | "</span></span></span>
</span>
</span>    #</span></span> Dump the specific metadata block</span>
</span></span>    dd</span></span> if=/dev/disk/by-id/DISK_SERIAL bs=1 skip=$</span>off</span></span> count=$</span>sz</span></span> ></span> metadata_$</span>off</span></span>.raw</span>
</span>
</span>    #</span></span> Parse the VHDs inside that block for comparison</span>
</span></span>    strings</span></span> metadata_$</span>off</span></span>.raw</span> |</span> grep</span></span> "</span>VHD-"</span></span></span> |</span> sed</span></span> '</span>s/ {.*//'</span></span></span> |</span> \
</span></span>    sort</span></span> ></span> vhds_$</span>off</span></span>.txt</span>
</span>done</span>
</span></code></pre>
Since I needed the entry to be precise to the byte, I used a block size of 1. This was not optimal but I wasn’t in the mood to do some bash arithmetic.</p>
Each LV had a VHD-</code> prefix. I used that to extract the names and count of LVs of each entry quickly.</p>
3. Differential Analysis</h3>
Now I had a bunch of text files representing different points in time. We knew from the console output of my disastrous lvremove</code> command exactly how many LVs were deleted (46) and the one that remained (1). Therefore, the pre-incident state should have 47 VHDs.</p>
#</span></span> 1. Sort by VHD count to find the candidate states.</span>
</span></span>wc</span></span> -</span>l</span> vhds_*</span>.txt</span> |</span> sort</span></span> -</span>rn</span></span> |</span> grep</span></span> '</span>47 '</span></span></span>
</span>
</span>#</span></span> 2. Verify the difference between our candidate (offset 5775360) and the current broken state</span>
</span></span>comm</span></span> -</span>13</span> vhds_deleted.txt vhds_candidate.txt</span>
</span>
</span>#</span></span> 3. Get the seqno of each candidate state and compare against the very next transaction</span>
</span></span>strings</span></span> metadata_candidate.raw</span> |</span> grep</span></span> seqno #</span></span> output: 347</span>
</span></span></span>strings</span></span> metadata_*</span>.raw</span> |</span> grep</span></span> 348</span>
</span>
</span>#</span></span> Check that the difference between the desired state (347) and the first deletion step (348) is exactly 1 VHD</span>
</span></span>comm</span></span> -</span>13</span> vhds_candidate.txt vhds_348.txt</span>
</span></code></pre>
This confirmed we had the exact offset (5775360</code>) right before the deletion started.</p>

	
		</i>Note</p>
	
In the case of this lvremove</code> command, each LV got deleted one by one. This meant one metadata entry per LV. Luckily there wasn’t enough LV to overflow the ring buffer.</p>

</blockquote>
4. The Restoration</h3>
With the correct raw metadata file isolated, we converted it into a standard LVM config file and did a dry-run test.</p>
#</span></span> Convert raw strings to config file</span>
</span></span>strings</span></span> metadata_5775360.raw ></span> recovery.vg</span>
</span>
</span>#</span></span> ALWAYS DO A DRY RUN FIRST</span>
</span></span>vgcfgrestore</span></span> VG_NAME --</span>test</span> --</span>file</span> recovery.vg</span>
</span>
</span>#</span></span> The real deal</span>
</span></span>vgcfgrestore</span></span> VG_NAME --</span>file</span> recovery.vg</span>
</span></code></pre>
5. Service Rescan</h3>
Once LVM was aware of the structure again, we rescanned the VG to make sure our storage driver could find each VHD.</p>
/usr/bin/vhd-util</span></span> scan -</span>f</span> -</span>m</span> VHD-*</span> -</span>l</span> VG_NAME</span>
</span></code></pre>
YES!!!</a> All LVs were restored and 100% of the data was intact. We rebooted the affected VMs and normal operations resumed roughly two hours after the incident started.</p>
Lessons Learned</h2>
Prod outages, besides being absolutely terrible are also incredible learning opportunities. Here is what I took away from this one:</p>

RTFM: The solution came from the technical specs and the man. I later found another blog post about this recovery method</a></li>
Do dry-runs before destructive commands: Doing a dry-run before anything could have prevented us from running the command with the wrong syntax.</li>
Beware of fatigue: It was the end of the day. A simple peer review of the command syntax, or a dry-run flag, would have prevented this entirely. I know I get lenient with commands I use all the time.</li>
Always backup your data: If the LVM archive/backups were properly configured we could have restored to the previous state quickly and make this a matter of minutes.</li>
</ol>
In the end, we saved the production environment without data loss. It was a stressful two hours, but the custom scripts and expertise gained made it a valuable learning experience.</p>
Handling stress</h2>
Watching the command delete each volume one by one triggered an intense wave of stress and a minor panic attack. I somehow managed to overcome it and became strangely calm and hyper-focused during the actual recovery process.</p>
It was only after the call ended that I truly understood the phrase “the silence is deafening.” Sitting alone, drowning in that silence, was the worst feeling of the entire afternoon. Fortunately, I already had dinner plans with some friends, so I rushed outside to get some fresh air and hit the road. I called one of my friends on the way because I really needed to talk it out. I managed to pull myself together later on and actually had a wonderful evening.</p>
Unsurprisingly, I am now the target of constant jokes whenever the topic of removing LVs or deleting data comes up. I honestly deserve it</p>

Mandatory part code explanation</h1>
Slightly inspired by debian glibc malloc</a></p>

Overview of my infrastructure

Introduction</h2>

Constraints</h2>
I don’t have a lot of money to put into electricity bills, so it was and is important that the hardware I chose is not too energy-intensive. Additionally, low-energy equipment generally produces less heat, and I don’t like it when it’s hot in my apartment.</p>

My infrastructure</h2>

Security</h2>

Deployment of key services</h2>
To lighten this article, I’ll detail the deployment of key tools (Traefik, Authentik, Portainer, Gitea) in another article. I’ll put the link here when it’s done.</p>

Millefeuille's blog

How I recovered from a wrong lvremove

Coder workspaces with XCP-ng and XOA

Rewriting malloc

Millefeuille's blog

How I recovered from a wrong lvremove

Coder workspaces with XCP-ng and XOA

Rewriting malloc

Bonus part</h1> Making mandatory functions thread-safe</li> Additional info print (mem dump, allocations history…) through flags or extra functions</li> De-fragmentation of the bins</li>

Overview of my infrastructure

Constraints</h2> I don’t have a lot of money to put into electricity bills, so it was and is important that the hardware I chose is not too energy-intensive. Additionally, low-energy equipment generally produces less heat, and I don’t like it when it’s hot in my apartment.</p>

My infrastructure</h2>

Security</h2>

Deployment of key services</h2> To lighten this article, I’ll detail the deployment of key tools (Traefik, Authentik, Portainer, Gitea) in another article. I’ll put the link here when it’s done.</p>

Constraints</h2>
I don’t have a lot of money to put into electricity bills, so it was and is important that the hardware I chose is not too energy-intensive. Additionally, low-energy equipment generally produces less heat, and I don’t like it when it’s hot in my apartment.</p>

Deployment of key services</h2>
To lighten this article, I’ll detail the deployment of key tools (Traefik, Authentik, Portainer, Gitea) in another article. I’ll put the link here when it’s done.</p>