In a Dutch oven or stockpot, combine broth, water, celery, onion chunk, and ham bone. Bring to a boil, then reduce heat to low. Cover the pot and simmer for about 1 - 1.5 hours.
Near the end of that cook time, you can prepare the dumplings. In a medium mixing bowl, stir together flour, baking powder, and salt. Add in the milk, stirring until a shaggy dough forms.
Turn the dough out onto a well-floured surface. Use a floured rolling pin to roll the dough out to 1/4-inch thickness. Use a pizza cutter to cut the dough into squares (roughly an inch or two in size).
Remove the ham bone to a cutting board. Let it cool enough to be handled easily. Cut or pull the ham off the bone, then discard the bone. Set the ham aside.
Use a slotted spoon to remove the pieces of celery and onion from the broth, then discard.
Add the dough pieces to the simmering broth and simmer for about 20-30 minutes, stirring occasionally, allowing the dumplings to cook and the broth to thicken to a gravy-like consistency.
Return the ham pieces to the pot and stir to combine. Add salt and pepper to taste, if desired, then serve and enjoy!