Silly tavern response length reddit. A place to discuss the SillyTavern fork of TavernAI.

Silly tavern response length reddit. SillyTavern is a user interface you can install on your computer (and Android phones) that allows you to interact with text generation AIs and chat/roleplay with characters you or the community create. So after some tinkering around I was actually able to get Kobold AI working on Silly Tavern. A place to discuss the SillyTavern fork of TavernAI. I usually stick with the preset "Carefree Kayra", AI Module set to Text Adventure. Step 0 is to do that. I'm also using kobaldcpp to run gguf files. it's not like those l1 models were perfect. Quality of writing ( ️) and descriptiveness (🏞️). Step 2 - Check the "Show "External" models (provided by API)" box. The only better models are probably 70b and above. If the have the API set up already, Make sure Silly Tavern is updated and go to the first tab that says "NovelAI Presets". Once you have SillyTavern open in your browser, connect SillyTavern to KoboldCpp as follows: Reply reply. They predict. Response length. Before anyone asks, my experimented settings areMax Response Length = 400Temperature=0. So really the only way is through hitting the 'continue' button as other person pointed out. Didn't work. 8 which is under more active development, and has added many major features. But, they can gain some extra point s (👍) for cost, context size, availability, average response length, and so on. Context (tokens): change this to your desired context size (should not exceed higher This will give the AI a basic idea of how you want its responses to look and can help train it to get used to generating responses of a certain length. i use max, but not recommend cuz need lot of additional tuning of all settings. Here is what I have tried: Changed temperature Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. 2 kudo per billion params times the reply length divided by 80. Kunoichi-7B by SanjiWatsuki has been my most solid pick. Assuming the users keeps their inputs short and under 50 Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. 9 Rep. the top 75% tokens in the list (whose probability gets up to this value). Pen. The responses are very verbose and long. And what comes after char's line? User's. Do small adjustments to your jailbreak Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. SillyTavern is a fork of TavernAI 1. Download one of the many quantized versions of LLama 3. I've been using Noro for a bit on mancer, it has 8192 context size. Honestly 500-1000 tokens is fine If responses are repetitive, make your rep. Mistral: Medium always maxxing out the Max Response Length with each generation. 5. Q5_K_S" as my model. Those with nvidia GPUs probably tear through those processing times. Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. Silicone maid is right up there for being good as well. We serve them to users in our app. pen. Slope: 0. In "Character Note" input prompt, add: [Unless otherwise stated by { {user}}, your next response shall only be written from the point of view of { {char}}. Never run Ooba, but i think there must same as in ST. Also, if it ever gives you a response that is way too long, you can always edit it down manually to a more appropriate length and then continue the conversation. 8Presence Penalty=0. I recently started using SillytavernAI using the poe API, but I have trouble understanding how that "Context Size Tokens", "Token Budget" or "Scan Depth" works. json. Set response length in SillyTavern to how long you want responses to be. You should only be the big personality traits that are needing to be emphasized for focus. I've noticed on Openrouter no matter what bot i choose it almost always cuts off the end of the message. I think it listens pretty heavily to your example dialogues. In KoboldCPP, the settings produced solid results. If they are too bad, manually change old replies and the model responses should improve. first of all, let's say you loaded a model, that has 8k context ( context is how much memory the AI can remember), first what you have to do is go to the settings (the three lines to the far left): 1. At this point they can be thought of as Some characters write real long responses that get cut off with a 300 token limit. In ST, I switched over to Universal Light, then enabled HHI Dynatemp. . Probably 4096 tokens, given what the config. This Kunocchini-7b-128k-test version has worked well for higher contexts -- 16K or higher for example. 59. Context Size or Context Window is like the AI's Limit Response length to 300. 1-0. Help with NovelAi max tokens (Response length) in the given UI at the sliders to for But the problem is this - the computer doesn't stop. Write a longer intro message (the message at the very top), should be 300 tokens or more imo. At this point they can be thought of as I would also add instructions in system prompt to emphasize short answers (role-playing default response says two paragraphs), cut the response length to 120-150, set the flag to remove incomplete sentences and occasionally manually update char's dialogue as when it starts increasing response length it will learn and keep giving longer responses. 2. In the bottom left menu, just click Continue. My PC is pretty beefy: CPU: i5-11600KF. I never had a problem with evil bastard characters and cruelty, to do this, it is enough to find a suitable prompt, which will bypass censorship and bullshit with morality. Just remember to use the Noromaid context and instruct prompts, as well as the recommended model settings, though maybe with the context length set to 32768 and the response length set to something higher than 250 tokens. Absolutely cinema. RAM: 32GB. 92 Top A: 0 Top K: 80 Typical Sampling: 1 Tail Free Sampling: 0. I couldn't get the model to do that. I've tried adding "Limit responses to 1-15 sentences. SupportAkali. Works best at low values such as 0. And as for the length of the answer, this is easily regulated again by the prompt itself and control by max response length in the settings. Added ability to enable user ID randomization for OpenAI API via config. Best. No matter what I do the responses are always about 1-2 sentences and around less 60-100 tokens. 75 it would limit the tokens to choose up to the sum of 75%, i. 2s and then before I can finish reading, it erases it and puts up another response, and then erases that, and this cycle continues. Some write shorter ones. Tokens are words, bits of words, punctuation base units of meaning to the AI. I always have trouble remembering how it exactly works, but if you select 0. Check out. I keep it at around 180. You can also use Instruct Mode (if you're not already) and put whatever your desired length is there, and use Author's Note to give instructions about desired response length. If you’re using openAI GPT as your api, click the top left menu, and change the Max Response Length (tokens). At this I'm able to run most 7b models at 8k context or better. PSA: How to connect to GPT-4 Turbo. The two most common reasons for me are short intro messages and example messages. My prompt specifies longer detailed responses and my character cards have long example paragraphs. 150 token length is baked into the NAI models. GPU: RTX 3080 (10GB VRAM) I've had the best results with KoboldCpp using "athena-v1. on top, there are Context (tokens) and Response (tokens): 2. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. The AI Horde is a FOSS cluster of crowdsourced GPUs to run Generative AI. Added "Universal" presets for KoboldAI and Text Completion that use Min P sampler. The orange dash line shows the cut-over point). It should pick up after 1-2 cuts. 8 which is under more active development and has added many major features. 14K subscribers in the SillyTavernAI community. This is the most stable and recommended branch, updated only when major releases are pushed. If your reply length is set to 80 tokens and the model is 33b, it should be 66 kudos. Add a Comment. Large models like ChatGPT or Claude will easily spit out responses that are 200 tokens each. Sillytavern doesn’t use the model directly, it just tells kobold what you type, formatted in a way that kobold and the model it’s running can understand. I've never been able to get the "Limit responses to X tokens" to work. Step 1 - Choose OpenAI as chat completion source, enter API key, and hit the "Connect" button. A small tax is applied to mitigate inflation from anonymous requests. You mention in your post that there are ways to potentially speed up responses. Also settings depend on the version you are using, AFAIK v4 uses chatml format and v3 alpaca. Keep the traits for the personality section to one to two words, using commas. I tried adding it in the system prompt, in various variations. It also seems to make it want to talk for you more. I am using Airboros-13b. However, the post that finally worked took a little over two minutes to generate. range and top p higher. Honestly, I just can't justify trying to force square Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. Toggle Multigen on in advanced formatting. Honestly, a lot of them will not get you the results you are looking for. Goliath-120b is the best model I've used. If you're running it locally, I recommend Min P between 0. 3. Goliath 120B. YOSHIS-R-KEWL • 7 hr. Cinema. Added sampler seed control for OpenAI API. e. For example: Temperature: 5, Min P: 0. Below that, CapyTessBorosYi-34b might be the best sub-120b model I've used. Follow the SillyTavern installation instructions. So it's still around that 110 or so max words per response. Respond length . For me though, I like medium sized responses like it gives. With the response length NovelAI will lock it to 150 tokens for a response. Making up one example entry that’s the length you want seems to help. LLM's are submitted via our chaiverse python-package. The creators of silly tavern or whoever writes the documentation forgot to inform users of the fact that in the most recent update of silly tavern, there is no slider for bypassing authentication when using openai type apis like lmstudio, because what you now have to do is enter "not-needed" for the api LLM Frontend for Power Users. Help. Give 5-10 samples of how you want the model to respond to you: length, writing style, etc. One thing that has kind of helped was putting specific instructions in Author's notes etc - [Write only three paragraphs, using less than 300 tokens] or whatever. If there is a value in sliding_window, say '4096' it means that the maximum context Which had frustated me, because I had restarted the server, changed token settings, tried to change the response tokens to "200" several times; and still nothing. Increased the "Response (length)" slider max value to 2k by default and 16k when using the unlocked context option. I've tried some other APIs. Just cut the reply to a desired length. I don't have much response to the section about Haruka, I just found it a bit funny considering the other similarities I saw. Step 3 - Under "OpenAI Model", choose "gpt-4-1106 In the prompt I usually type "your responses are strictly limited to x words" personally I go with 80 - 120 words. I believe 50-100 tokens should give you your desired output length, you can mess around with it. Is there any way to change… Mythomax doesnt like the roleplay preset if you use it as is, the parenthesis in the response instruct seem to influence it to try to use them more. It probably hit the cap of whatever you have set. For some people that might be nice, but for a fast-paced roleplay I would prefer to prompt for 2 paragraphs max. Works well enough. Mixtral seems to need really detailed prompts in order to give responses while other models perform better with less instructions. If you want better luck with that OOC command as well you can try putting it in the jailbreak section and turn on send jailbreak since Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. I've used the newer Kinochi but liked silicone maid better, personally. If your character uses example chats, make them longer or delete them. You have to load the model into the web server. There are good 70b models as well (Xwin and Lzlv get a lot of love), and some of the 20b models like Psyonic-Cetacean and Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. This will chain generations together until it reaches an appropriate stopping point. "]}} Is there a setting for this somewhere or should I Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. 37 Rep. However, in Silly Tavern the setting was extremely repetitive. Silly Tavern would send old messages from the same chat (upto context size. for ST max 512 if run in full compatibility mode. While 13b l2 models are giving good writing like old 33b l1 models. PhantomWolf83. But I can give you the settings that I use. if they're too long, lower your response length and min length if it's too short, raise your response length and min length, you may have to go back and edit a few messages for it to get on track. Top p is supposed to select tokens that sum up to x probability. 05, temperature at 1. So, specifically about the Noromaid-20b model, I only have to comment that it is seriously underrated. 25$ for a 13B model is crazy expensive and definitely not worth it. The settings didn't entirely work for me. It seems to work at least okay but it still freaks out sometimes. Honestly, I just can't justify trying to force square Yeah the max character length on the website is about 600. You connect sillytavern to the kobold server with the API URL in View community ranking In the Top 10% of largest communities on Reddit. 8Top P=1. ago. It depends on the model. If you put it low, it will select a lot less. Response length is too high. So a 100 word text should be roughly 125-150 tokens. I tried to write "Your responses are strictly limited to 100 words" in the system prompt, but it seems to ignore what I'm telling it to do. SillyTavern is being developed using a two-branch system to ensure a smooth experience for all users. 0 Will change if I find better results. The AI Horde is usable in ST and will never stop working. • 6 mo. So did increasing the 2 paragraphs to 3 paragraphs in the response part of instruct mode settings. In the response, don't overly lecture or act super mature, roleplay. " to JB, Main Prompt, and Author's note and nothing seems to work. View community ranking In the Top 10% of largest communities on Reddit. That basically just relies on Min P by negating Top P and Top K. If those old messages are bad they influence the reply. • 17 days ago. Produces more coherent responses but can also worsen repetition if set too high. The settings example (now): Context: 4096 Response: 16 (I didn't insert that number, it just returns to it everytime. Hi, i've been having a weird issue with Mistral Medium via OpenRouter. It's context isn't huge (only 4k I think), but with vector storage and summaries that's less of an issue. Radiant-Spirit-8421 • 9 hr. But you have it set to 350, that means there are 300 extra tokens the LLM needs to fill in. I'm using sillytavern with the option to set response length to 1024, cause why not. Ai Responses Too Big. Also, it sometimes doesn't write the response in the format I want (Actions between *'s and dialogue in normal text). 02 and 0. This RP focused quant of noromaid mixtral is absolutely superb, it's my current favorite. Response Length (tokens) or Amount to gen = this is how long output in tokens. Couple of solutions here: Increase response length. Sample dialogue is your friend. Think of kobold as a web server. Remember LLMs don't think, don't understabd. The ayami ERP rankings is possibly more useful in the context of sillytaven. 2048 - 500 - 200 = 1348 tokens left for chat history to serve as the 'memory' for the AI. Don't speak for {{user}}. 1. Do not seek approval of your writing style at im new to using kobold ai in silly tavern. Models not registered as popular cost and give less. 250(256) is enough in most cases. Char's actions and dialog tipically take, around 100 to 150 tokens. This is what works for me. 10 Rep. In the advanced formatting I selected Trim Incomplete Sentences. If it is the first few responses it is based on the intro message and the example messages. Oob was giving me slower results than kobald cpp. Response length: 300 Context size: 2048 Temp: 1. The most likely explanation is that your configuration defaulted trough the update. yaml. ) Go to files, then click config. • 5 mo. I say "Hi" and it immediately generates a response in 1. I have played around a bunch with the model parameters and have searched online and can't figure it out. I just started to use Mancer yesterday and these settings are awesome! So, I'm trying to use LLMs (Kobold, Ooba, etc) to fill the void, but I keep running into issues of quality or response time. If anything I'm glad you used one of my characters! As for the last bit about Kobold, seems fine to me, but me mentioning in the first place was more a nitpick than anything. But how much is that really? That will depend on how lengthy your chat input and the bot's responses are. json for the model has the line "max_position_embeddings": 4096,, which for most models correlates with the native context length. The rest of the character information and minor personality traits along with physical description can go into the character information. Normaid7b . For example, "roughly" could be one token, or two, like "rough-ly" More tokens will take more space in your context but should give you more detailed/complex characters in general. Start KoboldCpp, set the context slider to a higher value like 4096 or 8192 and select the gguf file as your model file. A token is generally 3/4-3/5 of a word. Yeah the max character length on the website is about 600. Also check your response length in the settings. Some characters write real long responses that get cut off with a 300 token limit. Thanks again,--Crow. MythoMax always uses the same length as previous responses. Our mission is to crowdsource the leap to AGI by bringing together language model developers and chat AI enthusiasts. Adding things like "write { {char}}'s next reply in short form " in the system prompt might help, as well as the "### Instruction: (reply size=short ComprehensiveTrick69. 85Frequency Penalty=0. 10. This was with the Dynamic Kobold from the Github. It will continue generating from that point. Every generation I do seems to completely fill up to my max response length (usually 300) which is weird, considering most other models i've used dont ALWAYS generate to the max limit. For the jailbreak I had the jailbreak prompt be: "[Structure The paragraphs correctly, don't have weird line breakings in the response. The only real problem with this sub is that the lowest tier (arguably the one with a value proposition) has too few tokens to be usable in roleplay chat even with very efficient entries. If that's more than the desired response length, it truncates the response to fit, but doesn't rethink what it was going to write. I'm getting this error: Kobold returned error: 422 UNPROCESSABLE ENTITY {"detail": {"max_length": ["Must be greater than or equal to 1 and less than or equal to 512. Check out the value of max_position_embeddings, that's the maximum context length of the model. Reply. sliding_window too, if the value is 'null' then the maximum context length is the value of max_position_embeddings. 2 or above (YMMV), low RepPen, Top P to the max, and Top K to the bottom. Could someone give me tips to configure these options. Okay. No_Rate247. release -🌟 Recommended for most users. At this point they can be thought of as completely independent programs. : 1. It is pretty decent. Limits the token pool by cutting off low-probability tokens relative to the top token. 🤔 A place to discuss the SillyTavern fork of TavernAI. One of those frankenmerges where the model creators just slammed a few other models together, it works really, really well. I'm using a 16k llama2-13b on a 4090. 04 is super good for me rn. I am using Mixtral Dolphin and Synthia v3. 7 Samplers Order: Repetition Penalty Top K Top A Tail Free Sampling Typical Sampling Top P Temperature They sometimes even exceed the response length I set up (366). Most 7b models are kinda bad for RP from my testing, but this one's different. ) (KoboldAi Horde as API, by the way) I don't know what happened Mostly utilizing increased context length tokens 2048 (about 10mins for response, 4096/20mins) and when I have time at 8192 (about an hour or so). This guide is for people who already have an OAI key and know how to use it. It doesn't end. To get longer response the only think you need to do is press continue. It wasn't doing that before. It is of little concern. 01, but can be set higher with a high Temperature. Range: 1024 Top P: 0. Im using the 'Merica!' Jailbreak for clewd, and it seems to be working fine, except the generationd sre far too lengthy. My issues is that the bot keeps giving me pages long responses, monologing instead of letting me reply, or doing 30 actions Also apparently PsyonicCetacean-20b is really good for dark stuff. I'm guessing it's like that because it's designed for collaborating on writing stories vs just general text generation / chat botting. At this point to guarantee the length you want, set the Token limit to your liking, I like 125 for example, and enable the " Trim Incomplete Sentences " in the third tab. Iirc ST using Novel has a theorical hard limit of 170 tokens, but you can use multigen to get longer responses, i put 150 tokens in both chunks fields and increase the tokens in Response Length to 300 accordingly, you must also disable Streaming since it conflicts with it. CHAI AI is the leading AI platform. fn nr iu qt ep fe re hp sg ib