Latent Variables Improve Hard-Constrained Controllable Text Generation on Weak Correlation

Hard-constrained controllable text generation aims to forcefully generate texts that contain specified constrained vocabulary, fulfilling the demands of more specialized application scenarios in comparison to soft constraint controllable text generation. However, in the presence of multiple weak correlation constraints in the constraint set, soft-constrained controllable models aggravate the constraint loss phenomenon, while the hard-constrained controllable models significantly suffer from quality degradation. To address this problem, a method for hard-constrained controllable text generation based on latent variables improving on weak correlations is proposed. The method utilizes latent variables to capture both global and local constraint correlation information to guide the language model to generate hard-constrained controllable text at the macro and micro levels, respectively. The introduction of latent variables not only reveals the latent correlation between constraints, but also helps the model to precisely satisfy these constraints while maintaining semantic coherence and logical correctness. Experiment findings reveal that under conditions of weak correlation hard constraints, the quality of text generation by the method proposed exceeds that of the currently established strong baseline models.

@article{Zhu2024,
  title     = {Latent Variables Improve Hard-Constrained Controllable Text Generation on Weak Correlation},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {6},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Weigang Zhu and Xiaoming Liu and Guan Yang and Jie Liu and Haotian Qi},
  doi       = {10.14569/IJACSA.2024.0150639},
  url       = {https://doi.org/10.14569/IJACSA.2024.0150639}
}