{"id":205,"date":"2024-05-17T20:08:46","date_gmt":"2024-05-18T02:08:46","guid":{"rendered":"https:\/\/www.redshirt1701.com\/?p=205"},"modified":"2025-04-06T19:11:50","modified_gmt":"2025-04-07T01:11:50","slug":"pimox-node-blew-up-rebuild-and-replace-the-node","status":"publish","type":"post","link":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/","title":{"rendered":"How To Rebuild and Replace a Dead Node in a Pimox Cluster"},"content":{"rendered":"\n

May 17th, 2024<\/p>\n\n\n\n

Tonight, I had a Pimox node die on me.  It looks like the microSD gave up the ghost.  Once I identified the issue, I thought I had to build a new node and I couldn’t reuse the node configuration.  I thought this would mean I’d have to restore the Virtual Machines from backup or reclaim their hard drives from my shared storage.  I was wrong. <\/p>\n\n\n\n

You can Replace a Node<\/h2>\n\n\n\n

Once a node dies, you can remove the node and replace it with a new one.  Using the same name, the new node will inherit the VMs and the VMs will boot up, etc.  In my case, since it was the microSD card and not the Pi itself, I used the same Rapsberry Pi with a fresh install of Proxmox. <\/p>\n\n\n\n

 You will find posts saying you can’t re-use the name or the ip address or bad things will happen.  This isn’t true.  As long as you don’t power on the original node with the original install of Proxmox<\/em>, you are fine.  So say the official instructions and the official instructions worked for me.<\/p>\n\n\n\n

How to remove a node:<\/h2>\n\n\n\n

https:\/\/pve.proxmox.com\/wiki\/Cluster_Manager#_remove_a_cluster_node<\/a>
Rebuild with a new install of Pimox onto the same pi.  Make sure you install a new copy of Proxmox VE to the pi, using the old one will cause issues.<\/p>\n\n\n\n

Add the node back:<\/strong><\/h2>\n\n\n\n

https:\/\/pve.proxmox.com\/pve-docs\/pve-admin-guide.html#pvecm_join_node_to_cluster<\/a>
The replacement node will pick up all the old node’s VMs, LXC, & Templates.  The assets will be there as if nothing had happened.  However, just like any computer abruptly powered down, you might have issues with the VMs, they were “turned off” without being shutdown properly.
Note – don’t forget to add a network bridge or your VMs won’t boot and will throw “error code 1”<\/p>\n\n\n\n

If you aren’t going to replace the Node:<\/strong><\/h2>\n\n\n\n

You can restore the VMs from backup to another node.
if you don’t have recent backups, and you are using shared storage, recover the vms by either:
Finding the config folder in \/etc\/pve\/nodes <\/em>and move the vm configs to another node
recover the VM by hooking the hard drives to a new, diskless VM.
https:\/\/forum.proxmox.com\/threads\/how-to-create-virtual-machine-with-existing-disk.26142<\/a><\/p>\n\n\n\n

\n

“If it doesn’t exist anymore, you need to create a new VM with the same settings and without any disks, but using the same ID as the disk has. Then run qm rescan –vmid <ID>. After that, the disk will show up as an unused disk for the VM and can be attached in the UI.”<\/p>\n<\/blockquote>\n\n\n\n

Remove the Node from the GUI<\/strong><\/h2>\n\n\n\n

Finally, removing the node from the CLI will not remove it from the GUI.  The configuration files for the node are still there.  To remove the node from the GUI, follow these instructions:<\/p>\n\n\n\n

\n

“Later we remove the node from Proxmox GUI. For this, we login to any active node. Then we check \/etc\/pve\/nodes<\/em> folder. And we check for the folder name of the removed node and remove it. Otherwise, the GUI displays the removed node in it.”
https:\/\/bobcares.com\/blog\/proxmox-remove-node-from-cluster\/<\/a><\/p>\n<\/blockquote>\n\n\n\n

Also, rather than deleting the folder, you can rename the folder so you can save the configuration files.<\/p>\n\n\n\n

Conclusion<\/h2>\n\n\n\n

These are just two ways to deal with a Pimox node that has died and either needs to be replaced or a new node added. We also discussed options of how to recover the VMs that were on the dead node.<\/p>\n","protected":false},"excerpt":{"rendered":"

May 17th, 2024 Tonight, I had a Pimox node die on me.  It looks like the microSD gave up the ghost.  Once I identified the issue, I thought I had to build a new node and I couldn’t reuse the node configuration.  I thought this would mean I’d have to restore the Virtual Machines from…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-205","post","type-post","status-publish","format-standard","hentry","category-raspberry-pi"],"yoast_head":"\nHow To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701\" \/>\n<meta property=\"og:description\" content=\"May 17th, 2024 Tonight, I had a Pimox node die on me.  It looks like the microSD gave up the ghost.  Once I identified the issue, I thought I had to build a new node and I couldn’t reuse the node configuration.  I thought this would mean I’d have to restore the Virtual Machines from...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\" \/>\n<meta property=\"og:site_name\" content=\"Redshirt1701\" \/>\n<meta property=\"article:published_time\" content=\"2024-05-18T02:08:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-07T01:11:50+00:00\" \/>\n<meta name=\"author\" content=\"Dylan\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@dpedersen\" \/>\n<meta name=\"twitter:site\" content=\"@dpedersen\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Dylan\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\"},\"author\":{\"name\":\"Dylan\",\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e\"},\"headline\":\"How To Rebuild and Replace a Dead Node in a Pimox Cluster\",\"datePublished\":\"2024-05-18T02:08:46+00:00\",\"dateModified\":\"2025-04-07T01:11:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\"},\"wordCount\":636,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e\"},\"articleSection\":[\"Raspberry Pi\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\",\"url\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\",\"name\":\"How To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701\",\"isPartOf\":{\"@id\":\"https:\/\/www.redshirt1701.com\/#website\"},\"datePublished\":\"2024-05-18T02:08:46+00:00\",\"dateModified\":\"2025-04-07T01:11:50+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.redshirt1701.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How To Rebuild and Replace a Dead Node in a Pimox Cluster\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.redshirt1701.com\/#website\",\"url\":\"https:\/\/www.redshirt1701.com\/\",\"name\":\"Redshirt1701 - What's On My Mind\",\"description\":\"Supplemental guides for the Raspberry Pi to fill in the gaps in regular tutorials and other stuff\",\"publisher\":{\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.redshirt1701.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e\",\"name\":\"Dylan\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/192.168.7.21:8000\/wp-content\/uploads\/2024\/05\/image.png\",\"contentUrl\":\"http:\/\/192.168.7.21:8000\/wp-content\/uploads\/2024\/05\/image.png\",\"width\":1071,\"height\":1051,\"caption\":\"Dylan\"},\"logo\":{\"@id\":\"https:\/\/www.redshirt1701.com\/#\/schema\/person\/image\/\"},\"sameAs\":[\"http:\/\/192.168.7.21:8000\",\"https:\/\/x.com\/dpedersen\"],\"url\":\"https:\/\/www.redshirt1701.com\/index.php\/author\/redshirtdkp\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/","og_locale":"en_US","og_type":"article","og_title":"How To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701","og_description":"May 17th, 2024 Tonight, I had a Pimox node die on me.  It looks like the microSD gave up the ghost.  Once I identified the issue, I thought I had to build a new node and I couldn’t reuse the node configuration.  I thought this would mean I’d have to restore the Virtual Machines from...","og_url":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/","og_site_name":"Redshirt1701","article_published_time":"2024-05-18T02:08:46+00:00","article_modified_time":"2025-04-07T01:11:50+00:00","author":"Dylan","twitter_card":"summary_large_image","twitter_creator":"@dpedersen","twitter_site":"@dpedersen","twitter_misc":{"Written by":"Dylan","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#article","isPartOf":{"@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/"},"author":{"name":"Dylan","@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e"},"headline":"How To Rebuild and Replace a Dead Node in a Pimox Cluster","datePublished":"2024-05-18T02:08:46+00:00","dateModified":"2025-04-07T01:11:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/"},"wordCount":636,"commentCount":0,"publisher":{"@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e"},"articleSection":["Raspberry Pi"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/","url":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/","name":"How To Rebuild and Replace a Dead Node in a Pimox Cluster - Redshirt1701","isPartOf":{"@id":"https:\/\/www.redshirt1701.com\/#website"},"datePublished":"2024-05-18T02:08:46+00:00","dateModified":"2025-04-07T01:11:50+00:00","breadcrumb":{"@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.redshirt1701.com\/index.php\/2024\/05\/17\/pimox-node-blew-up-rebuild-and-replace-the-node\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.redshirt1701.com\/"},{"@type":"ListItem","position":2,"name":"How To Rebuild and Replace a Dead Node in a Pimox Cluster"}]},{"@type":"WebSite","@id":"https:\/\/www.redshirt1701.com\/#website","url":"https:\/\/www.redshirt1701.com\/","name":"Redshirt1701 - What's On My Mind","description":"Supplemental guides for the Raspberry Pi to fill in the gaps in regular tutorials and other stuff","publisher":{"@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.redshirt1701.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/178b5651c1b98c2201e455772be8947e","name":"Dylan","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/image\/","url":"http:\/\/192.168.7.21:8000\/wp-content\/uploads\/2024\/05\/image.png","contentUrl":"http:\/\/192.168.7.21:8000\/wp-content\/uploads\/2024\/05\/image.png","width":1071,"height":1051,"caption":"Dylan"},"logo":{"@id":"https:\/\/www.redshirt1701.com\/#\/schema\/person\/image\/"},"sameAs":["http:\/\/192.168.7.21:8000","https:\/\/x.com\/dpedersen"],"url":"https:\/\/www.redshirt1701.com\/index.php\/author\/redshirtdkp\/"}]}},"_links":{"self":[{"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/posts\/205","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/comments?post=205"}],"version-history":[{"count":6,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/posts\/205\/revisions"}],"predecessor-version":[{"id":241,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/posts\/205\/revisions\/241"}],"wp:attachment":[{"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/media?parent=205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/categories?post=205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.redshirt1701.com\/index.php\/wp-json\/wp\/v2\/tags?post=205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}