Mamba-3

(together.ai)

75 points | by matt_d 3 days ago

2 comments

  • nl 1 hour ago
    I'm looking forward to comparing this to Inception 2 (the text diffusion model) which in my experience is very fast and reasonably high quality.
    • cubefox 14 minutes ago
      Mamba-3 is an architecture while diffusion is, I believe, a type of objective. So these are not mutually exclusive and therefore not comparable.
  • robofanatic 1 hour ago
    > Mamba-3 is a new state space model (SSM) designed with inference efficiency as the primary goal — a departure from Mamba-2, which optimized for training speed. The key upgrades are a more expressive recurrence formula, complex-valued state tracking, and a MIMO (multi-input, multi-output) variant that boosts accuracy without slowing down decoding.

    Why can’t they simply say -

    Mamba-3 focuses on being faster and more efficient when making predictions, rather than just being fast to train like Mamba-2.

    • arendtio 0 minutes ago
      I don't get the downvotes, as I had trouble understanding the intro as well. It seems it was written for a very specific audience.
    • esquire_900 1 hour ago
      This is sort of what their first sentence states? Except your line implies that they are fast in training and inference, they imply they are focusing on inference and are dropping training speed for it.

      It's a nice opening as it is imo

    • E-Reverance 1 hour ago
      The first sentence basically does though, no?
      • robofanatic 51 minutes ago
        Of course my only objection was the language. LLMs are now old enough to leave the jargon behind and talk in simple easy to understand terms.