AbstractPhila PRO

Screw it, I'll analyze it anyway. I might find an easy route.

It won't be 34 though, I'll prune a bunch and find the most aligned clips. Probably 3 will do for a proper prototype.

The x34 crossover was excessive noise, the continuity is very difficult to rationalize and form a cohesive anchoring for.

Predominantly the boundaries. I ran multiple scans on the COCO data and found huge deviances of composite differentiation, which means the manifold overall would be massive.

Something like trying to represent 800b params in a single 84m param space, which even the best of alignment geometric would require considerably more refined mathematics to even channel. The full process will require a multi-stage relational conjecture and interpolation between alignment positive, negative, and including a symbolic relational architecture just to make sense of the noise itself.

It could be done, and I can say for certain it could be done. It's just a matter of what the finished product between such a complex relational structure would look like, and how much information would be required to actually train it. The math lines up, the architecture lines up, but it does not have a reasonable horizon for the manifold.

Simply put, because of the boundaries, even if constricted, training 34 experts at runtime is beyond my scope. It would require full patch extraction, not just attenuation like CaptionBERT. CaptionBERT is a different form of differentiation that allows rapid pooled learning between multiple from the same family, while the x34 would require multiple pooled learning from adjacent families. Each family requires it's own patchwork size, it's own formulas for alignment based on that, and it's own specific attenuation principality based on the adjacent differentiation.

Possible yes, very possible. A true challenge and test for the architecture with a team and a series of experts, but I am but one researcher. I would be here for days figuring out how to attenuate patch14 to patch16 differences, just to yield little information based on the hypersphere alignment.

I've noticed some definite data overlap in the system - a percentage of the validation data has been trained, the R1 constraints aren't necessarily perfect in the current state. Even without the overlap, the early stage model can be directly trained to be R1@100% within 3 epochs, so this isn't the crucial fail point. The fail point is in the large scheme tests having potential overlap, and these cannot have overlap for the big train.

This overlap happened when I switched from the 200k set to the 500k set, so the full 12 million set will need a new validation target other than itself. I only ran it twice, but both times likely bled 20k images of mixed origin from the other, which likely is less than 8% or so bleedover but it's enough to taint the outcome.

I'll require another dataset to validate, something completely removed from the attribution and completely differentiated. I'm going to likely use my own dataset as validation, which is essentially a billion trash prompts that cannot simply be solved, and often make zero sense.

Even a small percentage of the validation data having been trained is enough for me to resort to extreme measures, not to mention the damn thing is reporting R1 100% all the time which is annoying me. I want to see a legitimate series of impossible combinations that cannot be represented, essentially garbage noise mixed with pure captions that the model has never learned from.

The model cannot easily solve these, which will give a perfect measure. I'd say maybe a million of these will be the best possible impossible goal.