I actually don’t remember off the top of my head, huh. The output chain is not the one I use often. I’d think nft add rule tracing filter output ip protocol icmp icmp type echo-reply meta nftrace set 1 would do it. Just make sure the priority is low enough again.
**beep ** bop.
- 2 Posts
- 73 Comments
farcaller@fstab.shto
Selfhosted@lemmy.world•Local homeserver suddenly not responsive to SSH but can be pingedEnglish
1·2 months agosystemd will try to save any kernel panics in pstore via systemd-pstore, if it’s enabled in your kernel. I’d check
/var/lib/systemd/pstoreand see if anything in there on the next boot.Can you elaborate on ping “running”? Do you get actual icmp replies coming back? Because there’s no code path I can imagine where a ping would cause hdd activity (on a normally running system).
If ssh times out (and you don’t do anything fancy with the firewall), then it’s not sshd dead, it’s sshd not being able to respond. Grab a tcpdump for dst port 22 from your local machine while RPi is stuck and see if you get any replies whatsoever or it’s just retransmits going into the void.
My first rough suspicion would be ram abuse. Something eats up all ram and the system locks up and semi-dies. Pstore would have OOMs. You could run a local script for telemetry recording too to see if ram use spikes up before the system gets unresponsive.
It is accepted just fine then. You might want to check the reverse, start in top of the output chain and trace the echo reply.
nftables offers a very decent debugging interface. First, you add a rule to trace the packet (a new chain with high priority works best). Usually I’d suggest to add those rules by hand instead of relying on declarative configs:
nft add table ip ping_trace nft 'add chain ip ping_trace prerouting { type filter hook prerouting priority -301; policy accept; }'Then you add the actual tracing rule there with the shape to match the incoming traffic:
nft 'add rule ip ping_trace prerouting icmp type { echo-request, echo-reply } meta nftrace set 1'now you can run
nft monitor traceand see the decisions made for the matching packets. Remember to delete the ping_trace table afterwards to clean up.
farcaller@fstab.shto
Selfhosted@lemmy.world•How do people handle authoritative DNS redundancy for their self-hosted workloads?English
3·2 months agoFirst on how to get an ASN: you can buy it for reasonably cheap from a LIR. Some will even toss a free /48 with that. Happy to offer names in private so that there’s no advertising. Expect a budget quoted above.
Once you have an ASN, you need to get an upstream - actually two as RIPE mandates at least two (otherwise why’d you need an ASN). Some LIRs would offer transit with ASN purchase. You can upstream via your ISP, if they allow you to (that’s very rare). Another option is a tunnel (there are free and paid ones) or a VM somewhere (some cloud providers offer to set up bgp with VMs they host). Generally, free ones are enough for basic stuff. Not much bandwidth and oftentimes IPv6 only, but you don’t pay anything either. Besides, you can ask around in various network related chats. Practically, I can offer ip transit with some marginally low burstable bandwidth, and that’s pretty common. You can look/ask around https://discord.gg/ipv6 for example.
For getting ASN to your homelab you’re looking at a tunnel option, most probably. Great if you have static ipv4 - allows you to use more common tunnels, but is still doable with a floating IP (e.g. check bgptunnel).
farcaller@fstab.shto
Selfhosted@lemmy.world•How do people handle authoritative DNS redundancy for their self-hosted workloads?English
2·3 months agoBGP anycast person here. If you have any presence in the region RIPE operates in then the pricing is within the homelab reach. ASN and a block of /48 would be about 70 EUR/year.
Alternatively, something like route64 would happily tunnel you IPs they announce for about 2 EUR/month.
I’ve used VM in my homelab for a while - it’s genuinely a smaller ram footprint, especially on the ingress path. You can even scale query separately if required.
I’d suggest VictoriaMetrics over pure Prometheus - better ram footprint is pretty useful nowadays.
You probably want to mention that some prominent cryptographers think this just astroturfing?
farcaller@fstab.shto
Selfhosted@lemmy.world•An LLM prompt for finding silently-broken backups in your homelab (full prompt + example output)English
13·6 months agoOr you could, idk, have some metrics. That’s a wild idea, of course. Who uses monitoring when you can just ask an LLM?
farcaller@fstab.shto
Technology@lemmy.world•Backblaze silently redefines 'unlimited' backups and users discover it's not backing up DB and OD — as firm leans heavier into AI storage services, changes could signal shift away from home backupsEnglish
4·7 months agoA second offsite NAS with your friend? That’s what I did when I grew out of my old synology. My new NAS capacity is noticeably impacted by things like frequent local snapshots but I don’t need to back those up remotely and it saves space.
While matter technically requires IPv6 there are non-confirming devices (glares at Hue) that will only do IPv4. It will work with most of the matter controller networks, because in the end it’s a conscious decision to disable IPv4 there.
You really want the ECC ram and the motherboard/cpu combo that supports it.
farcaller@fstab.shto
Selfhosted@lemmy.world•My thoughts shopping around for a wiki solutionEnglish
1·9 months agoIf your note’s type is JSON (or TW’s native dictionary), you can query it as such in filters.
My search problem is that I rely on metadata a lot. It’s natural for me to want a UI that renders machine readable metadata in a way that my brain can process and that requires rich querying capabilities.
I tried them all and, so far, TW wins for me, with orgmode being second close (I like orgmode in vim, but it had some fatal rendering flaws and I don’t feel like using emacs just for notes).
farcaller@fstab.shto
Selfhosted@lemmy.world•My thoughts shopping around for a wiki solutionEnglish
2·9 months agotiddlywiki has one of the most insane search engines from this list. They have a whole filters syntax that can express pretty much anything imaginable, no? I went back to TW from Obsidian because I was tired from Obsidian’s trivial search functionality.
farcaller@fstab.shto
Selfhosted@lemmy.world•SSH Client for Linux Desktop and Android - Alternative to TermiusEnglish
3·9 months agoLet’s untangle those problems. I have a similar setup so I just want to share some ideas to show that you don’t need to copy keys.
If I’m traveling or I wipe my device or get a new one, I would have to add the new key to many servers as authorized keys
If you oftentimes access ssh from untrusted systems you’re kind of in a bad spot to begin with. The best thing you can have is a yubikey on a keychain. Everything else means you leak secret material (a password or a key) to a machine you don’t inherently trust.
Also, I want a key backed up in case of disaster since all of my devices are in my home most of the time
Again, something that you can easily solve with a hardware key [in a safe]. But realistically, in case of a disaster a local shell password login should be good enough?
I’d recommend you to think about what attacks are you trying to prevent by using a shared private key. I’m not saying it’s a bad concept, inherently having it in your password manager (like 1Password that even has ssh-agent support) is pretty common. The problem with just the keys is that it’s non-trivial to expire them if needed. You might be indeed better off with some web based authentication that you can access from any place which would ask you secret questions/send you a text message or do whatever 2FA you deem sufficient and mint you a short-lived certificate for ssh.
farcaller@fstab.shto
Selfhosted@lemmy.world•SSH Client for Linux Desktop and Android - Alternative to TermiusEnglish
27·9 months agoNot an answer, but I’m curious: what’s wrong with just having several ssh keys, one per device?
farcaller@fstab.shto
Selfhosted@lemmy.world•Decreasing Certificate Lifetimes to 45 DaysEnglish
251·1 year agoYou can absolutely run your own CA and even get your friends to trust it.
farcaller@fstab.shto
Selfhosted@lemmy.world•How to propperly Ansible and selfhost without burning out?English
191·1 year agoMy fear is, that if i don’t document well or not use ansible, I will be hating my life once my server dies and I have to restore my data and also set um my services again in a few years.
I’ve been there plenty of times, you’re not alone. There are two solutions to that problem, really, and it boils down to the classic pet vs cattle.
- Everything is a pet
Pets mean you care about every server. If it breaks, it’s cheaper for you to fix it than redeploy. The overwhelming majority of your setup will be pets. Why? It’s simpler. Things don’t break that often, and when they do, it’s okay to be low-effort in fixing them.
Write docs for yourself, even if it’s just notes on the sequences of commands to run to redeploy things. You will thank yourself when the server finally dies in two years and you have notes on how to bring everything back.
- Everything is a cattle
Cattle means there’s no difference between server A and B. Everything is replaceable. Ultimately, whatever you run can run to the same extent in AWS, your basement NAS, or on your desk PC.
Cattle is also a lot of work. You will learn an excruciating amount of things about storage, networking, visualisation, workload scheduling, and such. And it’s easy to be demotivated because of how much there is to learn.
So take it easy. Concur that your hobby world is full of pets, but learn how to do the cattle approach at your leisure. You’ll realise that in every practical cattle setup, there are still pets, and that automating yourself from complexity only means you add layers of it somewhere else.

nixos, mostly. It has a very steep learning curve but if it breaks I’m just “whatever, rollback now, fix properly later”.