Over the last few years, the Deep Reinforcement Learning (DRL) paradigm has been widely adopted for 5G and beyond network optimization because of its extreme adaptability to many different scenarios. However, collecting and processing learning data entail a significant cost in terms of communication and computational resources, which is often disregarded in the networking literature. In this work, we analyze the cost of learning in a resource-constrained system, defining an optimization problem in which training a DRL agent makes it possible to improve the resource allocation strategy but also reduces the number of available resources. Our simulation results show that the cost of learning can be critical when evaluating DRL schemes on the network edge and that assuming a cost-free learning model can lead to significantly overestimating performance.
No Free Lunch: Balancing Learning and Exploitation at the Network Edge
Mason, Federico;Chiariotti, Federico;Zanella, Andrea
2022
Abstract
Over the last few years, the Deep Reinforcement Learning (DRL) paradigm has been widely adopted for 5G and beyond network optimization because of its extreme adaptability to many different scenarios. However, collecting and processing learning data entail a significant cost in terms of communication and computational resources, which is often disregarded in the networking literature. In this work, we analyze the cost of learning in a resource-constrained system, defining an optimization problem in which training a DRL agent makes it possible to improve the resource allocation strategy but also reduces the number of available resources. Our simulation results show that the cost of learning can be critical when evaluating DRL schemes on the network edge and that assuming a cost-free learning model can lead to significantly overestimating performance.File | Dimensione | Formato | |
---|---|---|---|
No_Free_Lunch_Balancing_Learning_and_Exploitation_at_the_Network_Edge.pdf
non disponibili
Tipologia:
Published (publisher's version)
Licenza:
Accesso privato - non pubblico
Dimensione
992.56 kB
Formato
Adobe PDF
|
992.56 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
2111.11912.pdf
accesso aperto
Tipologia:
Preprint (submitted version)
Licenza:
Accesso libero
Dimensione
187.55 kB
Formato
Adobe PDF
|
187.55 kB | Adobe PDF | Visualizza/Apri |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.