This paper measures whether large language models (LLMs) can produce network device configurations that are not merely syntactically valid but functionally correct β that is, configurations that achieve the intended forwarding, routing, and policy behaviour once loaded onto real devices.
Rather than scoring generated configuration text against a reference file, the authors evaluate correctness by function: they load the generated configurations and test the resulting behaviour, then characterise the failure modes that syntax-only checks miss.