The realization of scalable embodied intelligence in robotic swarms represents a fundamental challenge in robotics and artificial intelligence, hindered by the limitations of conventional decentralized multi-agent reinforcement learning approaches. This paper introduces a novel integrated framework for decentralized reinforcement learning that holistically addresses these challenges through three key innovations: a dynamic hypergraph convolutional communication protocol for bandwidth-efficient coordination, a hierarchical policy network with recurrent state estimation for managing partial observability and enabling scalability, and a federated learning-inspired training paradigm for enhanced robustness and sim-to-real transfer. Extensive experimental evaluation demonstrates that our approach achieves a 78% reduction in communication overhead compared to state-of-the-art baselines while maintaining superior task performance in swarms of up to 50 agents in simulation. Crucially, the framework exhibits remarkable robustness, showing only a 16–22% performance drop when transferred from simulation to reality, compared to 45–68% for baseline methods. Physical validation on a swarm of 15 nano drones confirms the practical efficacy of our approach, with an 85% success rate in dynamic target pursuit tasks. Statistical analysis confirms the significance of these improvements (p<0.001). These results collectively establish a new state-of-the-art for deploying scalable, communication-efficient, and robust embodied intelligence in robotic swarms.