Reverse engineering a GPT is amazing!

Not sure about the specifics, but I've heard some folks talking about using constitutional AI techniques to analyze publicly released language model checkpoints and infer aspects of their architecture, training data, loss functions, etc. Super fascinating area! Would love to learn more if anyone has good resources to share.