Towards Green IoT: Energy Conservation in Core IoT Network Using Network Coding and Soft Actor Critic

Summary

This study integrates network coding with soft actor-critic reinforcement learning to achieve energy-efficient routing in core IoT networks, demonstrating a 40% reduction in energy consumption with a 97% packet delivery ratio.

Abstract

The rapid expansion of internet of things (IoT) networks has intensified energy efficiency challenges in core IoT networks, where data routing places high demands on node energy. This paper proposes a hybrid model integrating network coding (NC) with soft actor-critic (SAC) reinforcement learning to address these challenges. NC minimizes redundant transmissions by combining packets, while SAC agents dynamically select energy-aware routing paths based on node state, link quality, and residual energy.

The system was implemented using the NS-3 simulator with a 5 × 5 grid topology of IoT nodes (512-byte packets, 100s runtime) under varying traffic conditions. Results demonstrate that NC + SAC achieves a 40% reduction in energy consumption, a 97% packet delivery ratio, 50% improvement in throughput, and an extended network lifetime compared to traditional routing and standalone approaches.

Key contributions include novel NC-SAC integration, comprehensive NS-3 validation across multiple metrics, and demonstration of learning-driven coded communication for sustainable IoT infrastructures. The hybrid approach balances reliability, energy preservation, and performance, offering a scalable framework for next-generation green IoT networks.

Introduction

The internet of things (IoT) has emerged as one of the most transformative technologies of the 21st century, seamlessly connecting billions of devices to enable smarter homes, cities, industries, and healthcare systems¹. With this unprecedented growth, however, comes a pressing challenge: the issue of energy consumption. IoT devices, especially those deployed in the core network, are often battery-powered or resource-constrained. As the density of devices and data traffic rises, energy efficiency becomes a crucial factor for ensuring the long-term sustainability of IoT infrastructures. In this context, designing intelligent communication and routing strategies that balance performance with energy preservation is of paramount importance.

Traditional routing methods in IoT networks often focus on shortest-path or reliability-centric approaches. While effective in basic scenarios, these methods frequently result in redundant transmissions and rapid energy depletion, especially in dynamic or large-scale deployments^2,3. To address these shortcomings, researchers have explored techniques such as network coding (NC), which reduces the number of transmissions by combining multiple packets into a single coded packet. NC has proven to improve throughput, reduce latency, and optimize bandwidth usage. However, while network coding is powerful in reducing communication overhead, it does not inherently account for node energy levels or adapt to fluctuating network conditions^4,5.

This is where machine learning-driven decision-making plays a vital role. Reinforcement learning (RL) provides a particularly robust framework for enabling IoT nodes to make adaptive decisions by interacting with the environment and learning from rewards. Among RL algorithms, the soft actor-critic (SAC) has gained prominence due to its ability to balance exploration and exploitation through entropy regularization⁶. Unlike conventional RL models, SAC encourages diverse action selection, preventing the system from converging too quickly to suboptimal strategies. For energy-limited IoT networks, this translates into smarter routing choices that adapt not only to link quality and delivery success but also to residual energy levels at each node.

Recent studies have increasingly focused on the role of reinforcement learning in optimizing IoT communication and energy management. For instance, Dev et al.⁷ employed Harris Hawks Optimization for energy-efficient IoT communication, while Tuong et al.⁸ explored deep reinforcement learning for hierarchical duplexing in wireless IoT systems. More recent works^8,9 highlight how advanced reinforcement learning methods, such as proximal policy optimization and distributed multi-agent frameworks, can further enhance adaptability in large-scale IoT environments.

This research presents a novel hybrid communication model that jointly combines network coding (NC) with the soft actor-critic (SAC) algorithm to achieve energy-aware intelligent routing in core IoT networks. Unlike prior approaches that use NC or reinforcement learning independently or that employ discrete-action deep RL models, this study utilizes SAC’s entropy-regularized continuous-action formulation to dynamically optimize routing decisions based on network state, including energy levels, link stability, and packet buffer status. Integrating NC enables reduced retransmissions and improved bandwidth utilization, while SAC ensures adaptive path selection to balance network load and conserve node energy. This dual mechanism introduces a new paradigm of learning-driven coded communication, achieving measurable gains in energy efficiency, throughput, packet delivery ratio, and network lifetime over existing methods.

We implemented and tested the proposed framework using the NS-3 simulation environment, deploying a 5×5 grid of IoT nodes under varying traffic loads. Packet size, distance, and transmission energy models were carefully defined to reflect realistic IoT conditions. Through these simulations, we evaluated key metrics including packet delivery ratio (PDR), throughput, network lifetime, and energy efficiency. The results are as follows: compared to traditional routing and standalone models, the NC + SAC hybrid system reduced energy consumption by almost 40%, improved throughput by 50%, and extended network lifetime significantly, all while maintaining a packet delivery ratio close to 97%.

Beyond the numerical improvements, the real strength of this work lies in its scalability and adaptability. As IoT networks continue to grow in size and complexity, solutions that adapt dynamically to changing conditions will be vital. This hybrid framework not only addresses the immediate issue of energy conservation but also sets the stage for autonomous, self-optimizing IoT systems that can sustain themselves in real-world deployments. Table 1 contains the performance logs for parameters like energy consumption, packet delivery ratio, network lifetime, and throughput. Table 2 gives a summary of the results drawn from the experiment. Furthermore, the integration of reinforcement learning into IoT networking demonstrates the potential of AI-driven protocols to reshape the way devices interact in resource-constrained environments.

Recent advancements in intelligent communication strategies further reinforce the need for adaptive and sustainable IoT routing frameworks^10,11,12. Traditional clustering-based or static routing protocols often fail to accommodate dynamic network challenges such as fluctuating link quality, varying node energy levels, and interference, which can lead to rapid performance degradation and early node failures. Modern reinforcement learning-based approaches have shown strong potential in addressing these issues by enabling IoT devices to dynamically learn optimal transmission strategies through continual interaction with the environment^10,11,12. At the same time, Network Coding has demonstrated its usefulness in reducing redundant transmissions and improving spectral efficiency in constrained networking environments^5,13,14. However, most existing studies evaluate these techniques independently rather than in combination. Therefore, integrating NC with a continuous-control reinforcement learning method like Soft Actor-Critic presents a more robust and intelligent communication mechanism capable of adapting to real-time changes while conserving energy^15,16. This hybrid paradigm not only strengthens the resilience and autonomy of IoT networks but also aligns with emerging research trends in green communication systems and machine-learning-driven wireless architectures for sustainable large-scale deployments.

Protocol

Data transmission and packet processing workflow
Node initialization: Each node starts with complete network topology information, its maximum energy capacity, and a pre-trained SAC agent ready for decision-making. The network coding module activates and prepares for packet combination operations.

Packet encoding process: When transmission begins, source nodes gather multiple data packets from sensors. The network coding module combines these packets into fewer coded packets using efficient mathematical operations, significantly reducing the total number of transmissions needed.

SAC routing decision execution: The SAC agent analyzes the current network state, including neighbor energy levels, link quality, traffic conditions, and destination distance. It selects the optimal next-hop neighbor based on its learned policy, prioritizing energy efficiency and reliability.

Multi-hop packet forwarding: Coded packets travel through the selected path. Intermediate nodes decide whether to further combine packets or forward them directly, based on their current buffer status and remaining energy levels.

Reward computation and storage: After each transmission, the system calculates a reward score based on delivery success, energy used, transmission delay, and packet drops. This reward information is stored for the SAC agent to learn from.

Continuous learning update: The SAC agent regularly reviews stored experiences and updates its decision-making strategy to become better at selecting energy-efficient routes while maintaining high delivery performance.

System architecture
A typical node operates as follows: it senses and collects data, encodes it using network coding, and then forwards the encoded packet through a routing path selected by the SAC agent. The SAC agent evaluates the node’s state, including remaining energy, buffer status, and link quality, to decide the next optimal hop. This process is dynamic and evolves through experience as the network conditions change. Figure 1 will illustrate this data flow and interaction between components.

Network coding equation
Network coding is applied at intermediate nodes as follows:

Where C_i is the encoded packet, α_ij is the coding coefficient (learned via SAC), and P_j are the original packets. This allows the network to reduce the number of transmissions and improve reliability.

SAC objective function
The SAC agent optimizes the policy using the following entropy-regularized objective:

Where r(s_t, a_t) is the reward, α is a trade-off parameter for exploration, and H is the entropy term.

Energy consumption model
To simulate realistic energy usage, the following model is used:

Where: E_elec- is energy per bit to run the transmitter/receiver, - k is the data size in bits, - is the energy required by the transmitter amplifier, - is the distance between nodes, -d is the path loss exponent.

Evaluation metrics
The following performance metrics were used in the simulations in this study:

Packet delivery ratio (PDR):

Throughput:

Network lifetime: Defined as the duration until 50% of the network nodes exhaust their energy.

Energy efficiency:

Pseudocode
Initialization of IoT network nodes
This function sets up all IoT nodes in the network before the system begins operation. Each node is assigned maximum energy, an empty buffer to store packets, a state representing its local environment, a SAC agent for learning optimal decisions, and an NC module that enables intelligent data encoding.

def initialize_nodes(network):
for node in network:
node.energy = MAX_ENERGY
node.buffer = []
node.state = sense_environment(node)
node.agent = initialize_sac_agent()
node.nc_module = activate_network_coding()

Packet generation and network coding
This function is responsible for collecting sensor data from the source node, encoding it using a linear or XOR-based network coding method, and preparing it for transmission. It reduces the communication overhead by bundling multiple packets into one coded packet, ready to be forwarded through the network.

def generate_and_code_packets(source_node):
data_packets = collect_data(source_node)
coded_packet = network_code(data_packets)
queue_transmission(source_node, coded_packet)

def network_code(packets):
return linear_combination(packets)

SAC-based routing decision
Here, the current node uses its SAC agent to analyze the network state and select the most energy-efficient and reliable neighbor to forward the packet to. The decision is based on learned policies and current environmental conditions, like link quality or residual energy.

def select_next_hop(current_node):
current_state = observe_network_state(current_node)
action = sac_agent_policy(current_node.agent, current_state)
next_hop = map_action_to_neighbor(action)
return next_hop

Packet transmission and feedback loop
This function manages the actual transmission of a coded packet. After determining the best next hop, the packet is sent if the link is valid. The system then calculates a reward based on transmission success and energy usage, which is used to update the SAC agent’s learning model.

def transmit_packet(current_node, packet):
next_hop = select_next_hop(current_node)
if is_link_valid(current_node, next_hop):
send(packet, next_hop)
reward = compute_reward(current_node, packet)
update_sac_agent(current_node, reward)
else:
recompute_next_hop(current_node)

SAC agent update (learning phase)
Once a packet is transmitted, this function updates the SAC agent. It records the change in state, stores the experience in memory, and if enough experience is gathered, it uses a sample batch to improve the agent’s decision-making strategy. This helps the node adapt to network changes over time.

def update_sac_agent(node, reward):
new_state = observe_network_state(node)
store_transition(node.agent.memory, node.state, node.action, reward, new_state)
if enough_experience(node.agent.memory):
batch = sample_batch(node.agent.memory)
node.agent = optimize_sac(node.agent, batch)
node.state = new_state

Decoding at the receiver
At the destination, this function takes in all the received coded packets and decodes them back into the original data using methods like Gaussian Elimination. The recovered data is then passed to the appropriate application layer or process.

def decode_packet(destination_node):
received_packets = destination_node.buffer
original_data = decode(received_packets)
deliver_data(original_data)

def decode(packets):
return gaussian_elimination(packets)

Energy monitoring and node sleep logic
This function tracks each node’s energy levels and determines whether a node should continue operating or switch to sleep mode to conserve power. If energy is sufficient, the node deducts transmission-related costs; if not, it shuts down temporarily to extend overall network life.

def monitor_energy(node):
if node.energy < THRESHOLD:
put_node_to_sleep(node)
else: node.energy -= transmission_cost()

Results

We conducted simulations comparing four different approaches: (i) A traditional model without optimization, (ii) A system employing only Network Coding (NC), (iii) A setup using only the soft actor-critic (SAC) algorithm for routing, and (iv) The proposed hybrid model combining both NC and SAC.

Energy consumption
The most significant improvement was observed in energy efficiency. Traditional IoT networks consume energy rapidly due to reliance on fixed routes and frequent r...

Discussion

This research introduces a novel energy-aware communication strategy for core IoT networks that integrates network coding (NC) with the soft actor-critic (SAC) reinforcement learning algorithm. The objective of this hybrid approach is to minimize energy consumption while maintaining reliable and efficient data transmission. NC reduces redundant transmissions by encoding data packets, while SAC dynamically learns and selects energy-efficient routing paths based on observed network states. The simulation results presented ...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
6 GB NVIDIA GeForce RTX 3050 (Computer system)	nVIDIA	https://www.nvidia.com/en-in/geforce/graphics-cards/30-series/rtx-3050/	For simulation and training: CPU RAM: 16 GB, GPU RAM
AMD Ryzen (Computer system)	AMD	Ryzen 7	For simulation and training: Processor
Hardware Testbed (optional future work)	Raspberry Pi	https://www.raspberrypi.com/products/raspberry-pi-4-model-b/	Raspberry Pi 4 nodes with wireless modules for real-world IoT deployment and validation
Intel Core i7 (Computer system)	Intel Core	i7-7700 Processor	For simulation and training: Processor
Jupyter Notebook	Jupyter	https://jupyter.org/	Environment for iterative development, debugging, and visualization
Matplotlib	Python	https://matplotlib.org/	Python visualization libraries for plotting performance metrics (Energy, Throughput, PDR, Network Lifetime)
Network Coding Module			Custom Python/NS-3 integration for implementing linear network coding and packet encoding
NS-3 Simulator	ns-3	nsnam.org	Open-source discrete-event network simulator used for IoT network simulation and routing evaluation
NumPy	NumPy	numpy.org	Python packages for handling simulation logs, statistical analysis, and structured datasets
Pandas		pandas.pydata.org	Python packages for handling simulation logs, statistical analysis, and structured datasets
Python 3.10	Python	Version 3.10.0	Programming language used for SAC agent training, coding coefficient optimization, and data processing
PyTorch	PyTorch	https://pytorch.org/	Deep learning library for implementing the Soft Actor-Critic algorithm and reinforcement learning models
Seaborn	Python	https://seaborn.pydata.org/	Python visualization libraries for plotting performance metrics (Energy, Throughput, PDR, Network Lifetime)
Ubuntu 20.04 (Computer system)	Ubuntu	Version 20.04	For simulation and training: operating system