Efficient Streaming Language Models with Attention Sinks
스트리밍 언어 모델의 효율성 향상과 Attention Sinks
Recent Posts
재미있는 프로그램
https://www.youtube.com/watch?v=v8O_tSF_o50
Mamba： Linear-Time Sequence Modeling with Selective State Spaces
Mamba: 단순화된 시퀀스 모델링
이미지 사이트들
https://paperswithcode.com/sota/image-classification-on-imagenet
[중요한 것은 꺾이지 않는 RL] Multi Environment Reinforcement Learning
https://www.youtube.com/watch?v=4_V6yosIDJc
필요 사이트 timm + torch
https://github.com/huggingface/pytorch-image-models?tab=readme-ov-file#features
GEM Pooling
``` class GeM(nn.Module): def init(self, p=3, eps=1e-6): super(GeM, self).init() self.p = nn.Parameter(torch.ones(1)*p) self.eps ...
Segforemer
참고
LORA
https://arxiv.org/abs/2106.09685
cal q learning
https://arxiv.org/abs/2303.05479