<<Prev Next>> Scroll to Bottom
Stuff goes here
L1[02:20:21] * Amanda meows to Elfi the secret coordinates for the Martian cache of Mars bars, does a heccen zzzmew
L2[02:21:04] <Amanda> Rogjt, so you go antinode to Phobos...
L3[02:23:45] <Amanda> Night girls, be good and maybe the goddesses will sprinkle some Essence in your sprouts in their gardens
L4[02:39:21] <Re​nno> anyone have any thoughts on using CBOR compression to keep a large amount of read-only data stored in memory? I wonder how much memory it actually saves
L5[02:39:59] <Hawk777> What do you mean by CBOR compression?
L6[02:40:41] <Re​nno> well, its in a more compact format data wise even though I know its not meant for "compression" like a compression algo would be
L7[02:40:48] <Hawk777> Ah. Well, compared to what, then?
L8[02:40:57] <Re​nno> lzss, for example
L9[02:41:17] <Hawk777> If your alternative is JSON, then CBOR might be smaller. Compared to just raw binary data where the code knows the offsets, probably larger.
L10[02:41:34] <Hawk777> Kinda depends on what the data is and what you need to do with it.
L11[02:41:42] <Re​nno> presently the autocrafting calculator I'm working on is deserializing an lzss compressed database file each time it needs to pull a recipe and it stores it in a limited cache
L12[02:42:10] <Re​nno> so it's quite slow since it has to do so many checks and each io.open takes a component call I think
L13[02:42:39] <Hawk777> I assume the database is too big to keep entirely in RAM?
L14[02:43:11] <Re​nno> the advantage being that memory impact is limited and size on hdd is maximized
L15[02:43:12] <Re​nno> and I do not know, the library for reading from the compressed databases is of Izayas design
L16[02:43:14] <Hawk777> In a “native” form I mean, like tables or whatever.
L17[02:44:14] <Re​nno> the sum of the database files is 1.32mb
L18[02:44:22] <Re​nno> idk how that translates into used memory in the OC environment
L19[02:45:39] <Hawk777> Well, normally when I think about a larger database-type file, I think of random access. A single bulk-compressed file is very much not random-accessible, normally; you have to read it from start to finish. Using a file format that lets you jump around to immediately access the data you actually care about, without having to load the whole file, should (at least it would on a real computer, OC may be a little different due to its compone
L20[02:45:39] <Hawk777> all cost scheme) give better performance without costing a lot of memory.
L21[02:47:59] <Re​nno> I mean all its really doing is pulling lines like this from the database
L22[02:47:59] <Re​nno> minecraft:book {"minecraft:paper","minecraft:paper","minecraft:paper","minecraft:string","tconstruct:pattern","tconstruct:pattern"}
L23[02:48:06] <Re​nno> theres just thousands of lines like that
L24[02:48:37] <Re​nno> so there is probably a temporary spike in memory and cpu usage as it is right now when it reads through the file to find a recipe that isn't in the cache already
L25[02:48:40] <Hawk777> That’s not a great file format.
L26[02:48:53] <Hawk777> I mean it’s simple, but not good for searching, nor compact.
L27[02:49:03] <Re​nno> which is why its lzss compressed
L28[02:49:09] <Re​nno> the compression seems to be 70-80%
L29[02:49:13] <Hawk777> Which makes it smaller, but still no good for searching.
L30[02:49:20] <Re​nno> right
L31[02:49:32] <Re​nno> hence, why I am pondering CBOR
L32[02:50:02] <Re​nno> have it decompress, read everything in the file, convert it into the cbor format, and on retrieval in the calculation convert it back to normal
L33[02:50:23] <Hawk777> So you’d store the CBOR in a file?
L34[02:50:42] <Re​nno> the cbor-formatted data? that would be in a table for random access
L35[02:50:49] <Hawk777> Then why CBOR it?
L36[02:50:55] <Hawk777> Why not keep it as Lua objects?
L37[02:50:57] <Re​nno> supposedly less memory
L38[02:50:59] <Hawk777> If it’s only going to live in RAM anyway?
L39[02:51:05] <Hawk777> Have you tested?
L40[02:51:13] <Re​nno> No, which I why I was asking
L41[02:51:15] <Hawk777> TBH the first thing I would do with a problem like this is string interning.
L42[02:51:38] <Re​nno> I know not of what you speak
L43[02:51:52] <Hawk777> I don’t *think* Lua does that automatically, but you could do it yourself. Basically, stop storing a dozen copies of the string “minecraft:paper”. Instead just store one of them.
L44[02:52:10] <Re​nno> and everything after that would be a pointer to the string?
L45[02:52:32] <Hawk777> I think even just reusing the same object will do the job.
L46[02:52:48] <Re​nno> well the string is stored to the variable and the variable would be the pointer?
L47[02:53:06] <Hawk777> I *think* that something like “x = read_some_string_from_file(); y = read_the_same_string_from_file()” will create two strings with the same text in them, but “x = read_some_string_from_file(); y = x” will create only one.
L48[02:53:27] <Re​nno> that would be a pointer would it not?
L49[02:53:29] <Hawk777> So you don’t need to invent pointers or indices or anything like that.
L50[02:53:34] <Hawk777> Sure, under the hood.
L51[02:54:30] <Re​nno> https://www.luafaq.org/gotchas.html#T9
L52[02:54:30] <Re​nno> ```Now, how does string equality work? Unlike Java, s1==s2 will work on equal strings, character by character. This is because there is only ever one instance of any particular string stored in memory, so comparison is very quick (this is called interning), for an extra cost of string creation. This can work because strings are
L53[02:54:30] <Corded> 'immutable', there is no way in Lua to modify the contents of a string directly.```
L54[02:55:14] <Re​nno> It appears to already be a thing
L55[02:55:30] <Hawk777> Well, if one can believe what that page says (that there really *IS* only ever one copy of a given string, and not merely that equals literals or whatever get interned which is much more common), then I guess so.
L56[02:57:25] <Hawk777> A very simple but possibly workable idea. What if you just store each recipe in a different file, named after the output item it creates? If you only ever need to search for recipes by output item name, then finding the exact recipe from the big set is just a matter of opening the file with the proper name.
L57[02:58:15] <Re​nno> that would be dramatically slower
L58[02:58:24] <Hawk777> OK.
L59[02:58:55] <Re​nno> each one of those would still be a component call and you'd have way more files for no reason lol
L60[02:59:18] <Re​nno> so you might *maybe* save on a little cpu time since the decompression wouldn't have to do as much
L61[02:59:39] <Hawk777> Well, if what you were already doing was reading the file for each query, then it wouldn’t be any *more* file-opens, but that one pass would do much less work.
L62[03:00:07] <Hawk777> If you’re already able to do multiple operations in a single pass through the file, then yes, it would be worse, because you’d have to open a file per operation.
L63[03:00:24] <Re​nno> well that it what I was working towards when I was told about CBOR
L64[03:00:25] <Hawk777> I don’t know what exactly the baseline is, so it was a suggestion, not necessarily and improvement depending on where you’re starting from.
L65[03:00:51] <Hawk777> The thing I don’t understand is, if you want to use CBOR, putting it in *memory* seems like a very weird place to do it. Why wouldn’t you convert the *file on disk* to CBOR instead?
L66[03:01:18] <Re​nno> that wouldn't have nearly as much compression?
L67[03:01:22] <Re​nno> as lzss
L68[03:01:23] <Hawk777> What?
L69[03:01:29] <Hawk777> I… guesS?
L70[03:01:53] <Hawk777> I must be very confused about your objective. I can’t seem to figure out whether your objective is to keep the entire database in memory, or to read what you need from the file as you need it.
L71[03:02:07] <Re​nno> cbor is made more for serialization/deserialization from what I understand, where lzss is specifically made for compression, the whole ponderance of using cbor would be simply to make an improvement on memory usage of storing objects
L72[03:02:09] <Hawk777> I was assuming the latter, since I thought you had too much data to keep the whole thing in memory.
L73[03:02:54] <Re​nno> it should be clear from my initial question that I would rather have it in memory but I the question is if its really worthwhile to have it in memory or not
L74[03:03:06] <Hawk777> I’m not even sure CBOR would actually make an object smaller though, not if you just serialize things in the most obvious way.
L75[03:03:08] <Re​nno> as the current implementation is already "in file"
L76[03:03:24] <Hawk777> There are many ways of keeping data in a file, some of which might be faster than others, but anyway.
L77[03:04:06] <Re​nno> sure, but the cpu time is pretty minimal, the real time sucker is the component call as I said
L78[03:04:08] <Hawk777> Your luafaq posting pointed out that strings are interned in Lua. Strings are not interned in CBOR. So if you CBOR-encode your example line, you’d end up with “minecraft:paper” encoded three times.
L79[03:05:01] <Hawk777> So keeping things as native Lua objects might gain more from string interning than it loses from a few pointers and table structures overheads.
L80[03:06:33] <Hawk777> There’d be only one Lua string object, but that object might be larger than a little handful of objects you’d have otherwise. And of course keeping things as native Lua objects, you can’t beat the performance for searching and accessing data that way.
L81[03:07:56] <Re​nno> https://paste.pc-logix.com/taradahete
L82[03:08:31] <Hawk777> CBOR is binary, so you should not be checking its length using the unicode library.
L83[03:08:50] <Re​nno> it's the same length either way..
L84[03:09:39] <Hawk777> What data does the encode produce?
L85[03:09:55] <Hawk777> I’m not sure how you expect to encode a table with string keys as an array.
L86[03:10:21] <Hawk777> That doesn’t seem like it should work.
L87[03:10:28] <Re​nno> It works
L88[03:10:50] <Hawk777> OK. Doesn’t change the fact that I would like to know how it works
L89[03:11:05] <Re​nno> https://github.com/Zash/lua-cbor
L90[03:11:18] <Re​nno> I don't really know myself
L91[03:11:20] <Re​nno> it just does
L92[03:11:34] <Hawk777> To quote: “Encodes a Lua table as a CBOR array. Uses ipairs internally”
L93[03:11:56] <Hawk777> You made a table with four elements; according to the documentation, by choosing to encode it as an array, you will only encode two of the four elements.
L94[03:12:18] <Hawk777> Have you decoded the encoded text and verified that it contains both the [foo], [bar], [1], and [2] keys?
L95[03:12:36] <Hawk777> According to the documentation, I would expect it to only contain [1] and [2].
L96[03:12:45] <Re​nno> https://paste.pc-logix.com/gubanuvafa
L97[03:15:55] <Re​nno> array is ipairs, map is pairs, ordered_map is pairs but no number indexes included
L98[03:16:38] <Hawk777> That’s exactly my point. You’re encoding the table using the array option. That means it uses ipairs. That means it will only exclude the [1] and [2] keys, it will just not include the [foo] and [bar] keys in your original table, they will not be in the encoded data.
L99[03:17:00] <Re​nno> right, btw I am coding this as I am talking so that has since changed
L100[03:17:06] <Re​nno> which is how I know what those do
L101[03:17:58] <Hawk777> The point I am making is, you gave an example of a table { foo = "hello", bar = "world", "foo", "bar" } which has two integer keys and two string keys, but by encoding it as an array, you are only including the integer keys, not the string keys. So sure it’s only 9 bytes long, but it’s also missing part of the data. So it’s not really a useful benchmark.
L102[03:18:32] <Re​nno> I know what you're saying, and as I just said, that has since changed, I am coding the benchmark as we speak
L103[03:18:37] <Hawk777> OK!
L104[03:20:16] <walks​anator> so wait how does OC persist external connections like tcp for example
L105[03:20:33] <walks​anator> and other network/handle related things
L106[03:20:59] <Hawk777> Files, IIRC it stashes the filename in the Value object which gets encoded to NBT, so when you reload the chunk, it reopens the file from disk.
L107[03:21:27] <Re​nno> https://paste.pc-logix.com/inezimupes https://discordembeds.pc-logix.com/live/1713842487319.PNG
L108[03:21:31] <Hawk777> Network connections I’ve never tried, I would guess it doesn’t, because that doesn’t make sense (how could you keep a network connection open when the entire Java VM process running Minecraft has ceased to exist, for example?).
L109[03:22:04] <Re​nno> ~16% smaller than normal serialization library and much faster
L110[03:22:23] <Hawk777> That makes sense, I think serialize does not intern strings either.
L111[03:22:48] <Re​nno> hard to really measure the memory though
L112[03:22:57] <Re​nno> probably will have to do that in ocelot
L113[03:23:08] <Hawk777> computer.freeMemory()?
L114[03:23:19] <Hawk777> Not sure how precise it is.
L115[03:23:41] <Re​nno> I have something to calculate the moving average of memory used but still
L116[03:24:10] <Re​nno> benchmarking having them stored in memory as a CBOR serialized string vs just having them in memory as normal tables is unclear
L117[03:24:50] <Hawk777> To get a good memory usage reading you might want to run garbage collection after loading the data. There might be a lot of temporary bits and pieces around that were used for loading but are not part of the permanent data.
L118[03:25:45] <Re​nno> and the alternative to this idea of having them stored in memory with CBOR and normal tables is storing the lzss compressed file contents in memory and selectively decompressing and reading as needed
L119[03:26:22] <Hawk777> Is LZSS a seekable format? Or would you have to start from the beginning of the file each time?
L120[03:26:31] <Re​nno> probably the beginning
L121[03:26:51] <Re​nno> unless I make another parser that makes a new format that is seekable (which is feasible)
L122[03:27:06] <Hawk777> If the compressed data fits in memory, and you can decompress it fast enough, that could work too.
L123[03:27:14] <Re​nno> right
L124[03:27:20] <Re​nno> the decompress time is pretty light
L125[03:27:41] <Hawk777> If the decompression is fast, maybe that would be cheaper than a component call to load the data from the file, so if LZSS lets you keep the data in RAM and none of the others do, that’s probably a win.
L126[03:27:56] <Re​nno> right
L127[03:30:41] <Re​nno> the overall compression of the db files from each mod is from ~1.28mb to 235kb
L128[03:30:54] <Re​nno> so in theory that'd be a pretty huge improvement on memory
L129[03:32:05] <Re​nno> though I guess if you really wanted speed, cbor is pretty good from the looks of it
L130[03:32:10] <Ocawes​ome101> @walksanator i expect tcp handles and whatnot are one of a very few things that don't get persisted.
L131[03:32:13] <Re​nno> might have to integrate that into my networking stack
L132[03:32:51] <Hawk777> Another idea to keep in the back of your head, if you *do* find that running the decompression is taking a bit too long. If you can afford enough memory to hold the item names, but not the recipes. You could cut up the database into, say, 10 parts, with an equal number of recipes in each part. Then also have a table mapping from item name to which part number contains the recipe for that item. When you need to access the recipe, now you
L133[03:32:53] <Hawk777> k that table and then decompress just one tenth as much data.
L134[03:32:57] <Hawk777> Of course tune the number 10 to taste.
L135[03:33:59] <Hawk777> If the part-number-lookup table and strings take up too much memory, you could (1) split on the colon, so that the mod-name part would get deduplicated; or (2) compress the entire lookup table, which would hopefully also be much smaller than the entire recipe database.
L136[03:40:19] <Forec​aster> %tonk
L137[03:40:19] <MichiBot> Fiddlesticks! Forec​aster! You beat Spider ​EveryOS's previous record of 3 hours, 28 minutes and 28 seconds (By 3 hours, 3 minutes and 35 seconds)! I hope you're happy!
L138[03:40:20] <MichiBot> Forecaster's new record is 6 hours, 32 minutes and 3 seconds! Forecaster also gained 0.01224 (0.00306 x 4) tonk points for stealing the tonk. Position #1.
L139[03:45:10] <Izzy> yeah that's what I did, I had the recipe files split up by mod
L140[03:45:17] <Izzy> not "equal" but good enough for a first pass
L141[03:47:11] <Izzy> if you wanted to get real fancy, hash the item name (simply!) and store based on the first N bytes of the hash
L142[03:57:14] <Forec​aster> %sip
L143[03:57:14] <MichiBot> You drink a mutable amethyst potion (New!). Forecaster turns into a lava girl until someone stops looking at them.
L144[04:05:46] <Re​nno> Well I am using the parser you made
L145[04:06:08] <Re​nno> Though it's quite confusing in places I have been making some upgrades here and there
L146[04:08:03] <Re​nno> I'm thinking the ideal searchable format should just be a precompiled dictionary of all the recipes where each index points to a list of recipes for the item?
L147[04:10:56] <Re​nno> Said dictionary, after decompress, could just be load()'d and indexed. I could maybe even cache the dictionary and dynamically uncache it if memory gets tight
L148[06:10:48] ⇨ Joins: Vexatos (~Vexatos@p200300eaef0455003661b6c45b192c84.dip0.t-ipconnect.de)
L149[06:10:48] zsh sets mode: +v on Vexatos
L150[07:03:45] ⇦ Quits: Hawk777 (~Hawk777@2001:569:7e40:1300:3285:a9ff:fe40:a36) (Quit: Leaving.)
L151[08:57:23] ⇦ Quits: Vexatos (~Vexatos@p200300eaef0455003661b6c45b192c84.dip0.t-ipconnect.de) (Ping timeout: 189 seconds)
L152[09:10:37] ⇨ Joins: Vexatos (~Vexatos@p200300EAEF09E1009491e8AA76fD5822.dip0.t-ipconnect.de)
L153[09:10:37] zsh sets mode: +v on Vexatos
L154[10:15:14] <jackie> I should really open Quassel more often. It taking about 5 minutes to sync up with the server usually means "you haven't looked at IRC for too long!" :P
L155[11:05:36] * Amanda shifts around some, meows and looks around
L156[11:05:53] <Amanda> Another busy night last night
L157[11:06:32] <Forec​aster> it would have been strange if it'd been a busy day last night
L158[11:07:22] <Forec​aster> or a busy day last night
L159[11:27:18] <Amanda> But it was a busy day (au) last night (est)
L160[11:48:04] <Forec​aster> damn timezones
L161[12:13:08] <Forec​aster> %tonkout
L162[12:13:08] <MichiBot> Darn it! Forec​aster! You beat your own previous record of 6 hours, 32 minutes and 3 seconds (By 2 hours and 46 seconds)! I hope you're happy!
L163[12:13:09] <MichiBot> Forec​aster has tonked out! Tonk has been reset! They gained 0.008 tonk points! plus 0.014 bonus points for consecutive hours! Current score: 1.5752, Position #1
L164[12:28:35] <Amanda> %remindme 20m check CI
L165[12:28:35] <MichiBot> I'll tell you "check CI" in 20m at 04/23/2024 12:48:35 PM
L166[12:48:35] <MichiBot> Amanda REMINDER: check CI
L167[13:27:31] <Forec​aster> %sip
L168[13:27:31] <MichiBot> You drink a serif nectar potion (New!). Tonk moved back 1 hour. (Rem. uses: 0)
L169[13:27:40] <Forec​aster> hooray
L170[13:27:56] <Forec​aster> except there is no tonk currently, so it didn't do anything
L171[13:28:00] <Forec​aster> but otherwise, hooray
L172[13:29:00] <Amanda> %splash @Forecaster with mutable honey potion
L173[13:29:00] <MichiBot> You fling a mutable honey potion (New!) that splashes onto @Forecaster. @Forecaster turns into a platypus girl until someone turns off a lamp.
L174[13:29:16] <Amanda> egg-laying mammal again! :D
L175[13:29:28] <Forec​aster> 😐
L176[13:30:11] <Forec​aster> but I hate having a tail, it's weird!
L177[14:16:02] <Spider ​EveryOS> %greenshell Vaur
L178[14:16:03] <MichiBot> Spider ​EveryOS: You hit Vaur! They lost 0.214602 tonk points which you gain! Congratulations! Position #3 Need 0.286136 more points to pass Va​ur!
L179[14:16:16] <Spider ​EveryOS> Yea!
L180[15:19:17] <Corded> > <Spider ​EveryOS> %greenshell Vaur
L181[15:19:18] <MichiBot> Spider ​EveryOS: You can't snipe right now. Try again in 22 hours, 56 minutes and 44 seconds.
L182[15:19:18] <Va​ur> you would have gotten more had you targetted Forecaster 😄
L183[15:19:54] <Va​ur> oh damn, fascinating, I can waste all your green shell that way xD
L184[15:24:48] <Corded> > <Va​ur> oh damn, fascinating, I can waste all your green shell that…
L185[15:24:48] <Spider ​EveryOS> IRC logs exist, you know. Can you please not impersonate me to manipulate the bot?
L186[15:25:34] <Va​ur> I'm only jesting, though someone should probably look into that
L187[15:25:49] <Spider ​EveryOS> Yea, definitely
L188[15:43:12] <Amanda> You should drain @Forecaster's shells so they finally fix it. :3
L189[15:43:51] <Amanda> or maybe I'll do another sneaky untested github drive-by
L190[15:44:05] <Amanda> if message.charAt(0) == '>' && isBot { return }
L191[15:44:27] <Amanda> Let the ~~goddesses~~ CI sort it out
L192[15:44:52] <Mim​iru> I SWEAR, the command parser USED to check for a prefix...
L193[15:45:17] <Amanda> Forecaster sometimes blows away upstream changes when they do big changes
L194[15:45:20] <Mim​iru> Who knows, maybe that got broken when I nuked MichiBot that one time.
L195[15:54:28] <Forec​aster> Michibot2 will
L196[15:57:23] <Forec​aster> I know I added a way to ignore suffixes at some point, pretty sure prefix is in there too
L197[15:57:43] <Forec​aster> Suffix was a config setting I remember
L198[16:16:57] <Amanda> Goddesses, I missed a word in reading this title "The END of CENTOS matters more than you" ... "think"
L199[16:26:29] <Forec​aster> The end of centos is the end of all
L200[16:30:13] <Corded> > <Forec​aster> Michibot2 will
L201[16:30:13] <Mim​iru> MichiBot 2fast 2furious
L202[16:40:57] <Michiyo> %restart
L203[16:40:58] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) ()
L204[16:41:07] <Michiyo> Let's see if I'm a moron.
L205[16:41:19] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L206[16:41:19] zsh sets mode: +v on MichiBot
L207[16:41:28] <Michiyo> %test
L208[16:42:00] <MichiBot> Mic​hiyo: Success
L209[16:42:01] <Michiyo> > %test
L210[16:42:02] <MichiBot> Mic​hiyo: Success
L211[16:42:10] <Michiyo> It seems I am.
L212[16:48:54] ⇨ Joins: ben_mkiv (~ben_mkiv@200116b815766a003038e7937bbd1d1A.dip.versatel-1u1.de)
L213[16:50:25] ⇨ Joins: Webchat259 (webchat@5.42.213.139)
L214[16:51:31] ⇦ Quits: Webchat259 (webchat@5.42.213.139) (Client Quit)
L215[16:51:36] <Michiyo> %test *
L216[16:51:37] <MichiBot> Mic​hiyo: Success
L217[16:51:42] <Michiyo> %test *%*
L218[16:51:42] <MichiBot> Mic​hiyo: Success
L219[16:51:46] <Michiyo> ... dafuq
L220[16:54:17] <Michiyo> %restart
L221[16:54:17] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) ()
L222[16:54:35] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L223[16:54:35] zsh sets mode: +v on MichiBot
L224[16:55:11] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Read error: Connection reset by peer)
L225[16:55:31] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L226[16:55:31] zsh sets mode: +v on MichiBot
L227[16:55:40] ⇨ Joins: MichiBot1 (~MichiBot@heimdall.pc-logix.com)
L228[16:55:40] zsh sets mode: +v on MichiBot1
L229[16:55:48] ⇦ Quits: MichiBot1 (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L230[16:55:51] <Michiyo> damn it
L231[16:56:28] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Remote host closed the connection)
L232[16:56:48] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L233[16:56:48] zsh sets mode: +v on MichiBot
L234[16:56:53] <Michiyo> WTF?
L235[16:57:30] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L236[16:57:35] <Michiyo> Oh.
L237[16:57:56] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L238[16:57:56] zsh sets mode: +v on MichiBot
L239[16:58:37] <Michiyo> %part #mekanism
L240[16:58:40] <MichiBot> Left channel #mekanism
L241[16:59:12] <Michiyo> %test
L242[16:59:13] <MichiBot> Mic​hiyo: Success
L243[16:59:19] <Michiyo> > %test
L244[16:59:19] <MichiBot> Mic​hiyo: Success
L245[16:59:27] <Michiyo> Ignored '> %test' because it starts with '>'
L246[16:59:28] <Michiyo> AND YET
L247[17:01:07] <Michiyo> I wonder....
L248[17:01:13] <Michiyo> %test [Edited]
L249[17:01:26] <Michiyo> *THAT* worked?
L250[17:02:58] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) ()
L251[17:04:42] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L252[17:04:42] zsh sets mode: +v on MichiBot
L253[17:05:56] <Michiyo> > %test
L254[17:05:57] <MichiBot> Mic​hiyo: Success
L255[17:06:10] <Michiyo> %test [Edited]
L256[17:06:27] <Michiyo> Yeah, it seems I'm a moron or something
L257[17:07:55] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L258[17:08:03] <Michiyo> https://paste.pc-logix.com/eriluraquw.js VS https://paste.pc-logix.com/mibuvayuko.js
L259[17:08:45] <Michiyo> But I'm doing the EXACT same thing https://github.com/PC-Logix/LanteaBot/blob/master/src/main/java/pcl/lc/irc/AbstractListener.java#L76-L86
L260[17:09:30] <Michiyo> waaait.
L261[17:10:37] <Michiyo> ... this is in GenericEventListener... not AbstractListener.
L262[17:10:53] <Amanda> Does it need to be done in https://github.com/PC-Logix/LanteaBot/blob/master/src/main/java/pcl/lc/irc/AbstractListener.java#L110 as well?
L263[17:11:29] <Amanda> Looks like a lot of the command parsing stuff is duplicated there
L264[17:11:41] <Michiyo> It seems to need to be done here: https://github.com/PC-Logix/LanteaBot/blob/4c0fbde894d773b9b8a37563370d0039f90343f1/src/main/java/pcl/lc/irc/hooks/GenericEventListener.java#L42-L51
L265[17:11:53] <Michiyo> As that is where I'm getting the "ignored because it ends with" print from
L266[17:12:27] <Amanda> ah
L267[17:14:16] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L268[17:14:16] zsh sets mode: +v on MichiBot
L269[17:14:24] <Michiyo> %test
L270[17:14:54] <MichiBot> Mic​hiyo: Success
L271[17:14:56] <Michiyo> > %test
L272[17:14:56] <MichiBot> Mic​hiyo: Success
L273[17:15:01] <Michiyo> .-.
L274[17:15:11] <Amanda> > %blame @inari somehow
L275[17:15:12] * MichiBot blames @inari somehow for Inari's lewdness!
L276[17:15:27] <Amanda> ... but that makes sense, MichiBot
L277[17:15:49] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L278[17:15:52] <Michiyo> lol
L279[17:16:36] <Michiyo> I REALLY wish I could locally debug current michibot
L280[17:16:56] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L281[17:16:56] zsh sets mode: +v on MichiBot
L282[17:17:14] <Michiyo> %test
L283[17:17:34] <MichiBot> Mic​hiyo: Success
L284[17:17:41] <Michiyo> > %test
L285[17:17:41] <MichiBot> Mic​hiyo: Success
L286[17:17:49] <Michiyo> I... don't even fucking know anymore.
L287[17:20:15] <Michiyo> OH lmao..
L288[17:21:27] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L289[17:21:46] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L290[17:21:46] zsh sets mode: +v on MichiBot
L291[17:21:52] <Michiyo> %test
L292[17:22:24] <MichiBot> Mic​hiyo: Success
L293[17:22:27] <Michiyo> > %test
L294[17:22:35] <Michiyo> Ok. there
L295[17:22:36] ⇦ Quits: MichiBot (~MichiBot@heimdall.pc-logix.com) (Client Quit)
L296[17:22:41] <Amanda> What was it?
L297[17:22:51] <Amanda> oh
L298[17:23:01] ⇨ Joins: MichiBot (~MichiBot@heimdall.pc-logix.com)
L299[17:23:01] zsh sets mode: +v on MichiBot
L300[17:23:04] <Michiyo> The change did have to be made in generic, but it helps if you swap endsWith with startsWith
L301[17:23:53] <Amanda> yeah...
L302[17:24:12] <Amanda> %splash @Forecaster with mutable currentcorn potion
L303[17:24:12] <MichiBot> You fling a mutable currentcorn potion (New!) that splashes onto @Forecaster. @Forecaster turns into a crocodile for 10 seconds.
L304[17:26:22] <Forec​aster> Hey D:<
L305[17:27:16] <Amanda> :3
L306[17:42:51] <Va​ur> %tonk
L307[17:42:52] <MichiBot> Bingo! Va​ur! You beat Forec​aster's previous record of <0 (By 6 hours, 29 minutes and 42 seconds)! I hope you're happy!
L308[17:42:53] <MichiBot> Vaur's new record is 6 hours, 29 minutes and 42 seconds! Vaur also gained 0.0065 tonk points for stealing the tonk. Position #2. Need 0.298602 more points to pass Forec​aster!
L309[17:43:08] <Va​ur> I've come back from worst
L310[18:51:07] ⇦ Quits: lunar_sam (c44a7f2987@jabberfr.org) (Ping timeout: 183 seconds)
L311[18:52:02] ⇨ Joins: lunar_sam (c44a7f2987@2a00:c70:1:178:170:40:189:1)
L312[18:53:40] <Amanda> stephan48: okay, I'll ask you before deciding on my old hack. Setting back up log injestion from my k3s cluster, and I'd like to avoid running the log collector as root, any idea how to set the mode/group of the files in /var/log/pods/ to something other than root:root:550?
L313[18:54:08] <Amanda> I thought maybe I could (ab)use systemd and use (Bind)?ReadOnlyPaths but that doesn't seem to be working
L314[18:54:40] <Amanda> ( My old hack being adding the group "root" to SupplimentaryGroups )
L315[18:54:41] <stephan48> no idea atm.
L316[18:55:00] <stephan48> huh
L317[18:55:22] <stephan48> you could do a volume mount and CAP_somethingsomethingDAC_READsomethingsomething
L318[18:55:49] <stephan48> which would allow you to skip permission checks?
L319[18:56:02] <stephan48> i think you talked about setting this up for borgbackup some time ago?
L320[18:56:33] <Amanda> The log collector is running on the host, as a non-root user
L321[18:56:48] <Amanda> k3s seems to be putting the logs as mode 550 owned by root:root
L322[18:57:15] <Amanda> the directories above the final log file are all 555 but the actual log files are 550
L323[18:57:27] <Amanda> ( I think it's 550 anyway, r-x )
L324[18:58:13] <Amanda> rather, 640
L325[18:58:20] <Amanda> "-rw-r----- 1 root root 1028 Apr 9 09:44 1.log"
L326[18:59:53] <stephan48> https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#TemporaryFileSystem= set to / and BindReadOnlyPaths= together with the Capability options
L327[19:00:03] <stephan48> never used it but sounds like a fun hack :D
L328[19:04:18] <Amanda> ... that's terrible, I'm trying it
L329[19:05:01] <Amanda> Also it's CAP_DAC_READ_SEARCH
L330[19:20:08] <stephan48> as i as saying :P
L331[19:22:42] <Amanda> Also I had to do "TemporaryFileSystem=." instead of "TemporaryFileSystem=/" because for some reason / was causing a EXEC exit code
L332[19:23:49] * Amanda tries adding the path of the executable to ExecPaths
L333[19:30:31] <Amanda> oh. It's a dynamic executable, I wonder if this'll work
L334[19:37:58] <stephan48> mh.
L335[19:38:01] <Amanda> oh bah
L336[19:38:22] <stephan48> TemporaryFileSystem=. will probably still allow quite alot of access to other parts of the fs :/
L337[19:47:11] <Forec​aster> %sip
L338[19:47:13] <MichiBot> You drink an aligned redstone potion (New!). Forecaster knows the exact location of a particular molecule of oxygen for 2 moons.
L339[19:54:53] <Amanda> bah
L340[19:55:06] <Amanda> TemporaryFileSystem=/ may not work after all
L341[19:55:19] <Amanda> The log scraper is also pulling metrics for the node as well
L342[19:55:24] <Amanda> including stuff like disk usage
L343[19:55:57] <Amanda> Looks like passing through /proc and /sys isn't enough to make sure that can be scraped
L344[19:57:18] <Amanda> With the TemporaryFileSystem=/ it just shows two mounts / and /run
L345[19:57:21] <Amanda> both as empty
L346[20:06:12] <stephan48> meh
L347[20:09:55] <Amanda> yeah, I'm just going to go back to my old hack of adding root to SupplimentaryGroups for meow
L348[20:10:06] <Amanda> I'll have to look into making k3s store the logs as non root:root sometime
L349[20:12:38] <Amanda> I tried chowning /var/log/pods once and it didn't seem to apply to new pods
L350[20:12:51] <Amanda> but I'm not sure if I'm doing the linux stuff wrong
L351[20:14:25] <Amanda> I'd think "chown -R root:foo /var/log/pods" would make new files be owned by root:foo, but maybe I just don't understand linux permissions as much as I thought I did
L352[20:58:19] <Spider ​EveryOS> College finals week is never fun, too much work and worry ):
L353[20:58:31] <Spider ​EveryOS> Currently in finals week at my colelge
L354[21:00:04] <Amanda> Ugh
L355[21:00:07] <Amanda> I need to gather spoons
L356[21:00:19] <Amanda> gotosocial is doing something weird when a user is deleted
L357[21:01:59] <Amanda> It's triggering a 500 statusCode response, when I would think a 404/403 would be more sensical for getting an update from a deleted user
L358[21:02:13] <Amanda> my monitoring is going nuts with alerts from this
L359[21:02:54] <Corded> > <Ama​nda> I need to gather spoons
L360[21:02:54] <Spider ​EveryOS> What does "gather spoons" mean
L361[21:03:04] <Amanda> Google "Spoon Theory"
L362[21:03:19] <Spider ​EveryOS> Thanks
L363[21:49:08] <Brisingr​ Aerowing> Asie: It seems that the OC 1.8.5 file never got approved by CurseForge.
L364[21:49:26] <Brisingr​ Aerowing> As the latest there is 1.8.4
L365[22:05:55] <Corded> > <Spider ​EveryOS> College finals week is never fun, too much work and worry ):
L366[22:05:55] <Re​nno> that's why I mess with OC stuff instead of doing schoolwork
L367[22:07:52] <The Pa​tmann💖> Now that sounds like a plan
L368[22:08:23] <The Pa​tmann💖> Galaxy Brain move if you can somehow relate the two
L369[22:19:20] ⇦ Quits: lunar_sam (c44a7f2987@2a00:c70:1:178:170:40:189:1) (Ping timeout: 183 seconds)
L370[22:20:01] ⇨ Joins: lunar_sam (c44a7f2987@jabberfr.org)
L371[22:21:17] <Izzy> big brain is writing a network stack during your networking course
L372[22:24:34] ⇦ Quits: xal (~xal@mx1.xal.systems) ()
L373[22:26:02] ⇨ Joins: xal (~xal@mx1.xal.systems)
L374[22:28:28] <Kristo​pher38> so if I'm doing formal verifiation course i'm supposed to write a theorem prover?
L375[22:32:44] <Amanda> %remindme 13h 18:28
L376[22:32:44] <MichiBot> I'll tell you "18:28" in 13h at 04/24/2024 11:32:44 AM
L377[22:46:09] <Corded> > <Re​nno> that's why I mess with OC stuff instead of doing schoolwork
L378[22:46:09] <Spider ​EveryOS> That's what I was doing on Sunday. But on Monday I realized I had to be more serious...
L379[22:54:44] ⇦ Quits: Vexatos (~Vexatos@p200300EAEF09E1009491e8AA76fD5822.dip0.t-ipconnect.de) (Quit: Insert quantum chemistry joke here)
L380[23:18:27] <Re​nno> some promising results,
L381[23:18:27] <Re​nno> quite a few recipes loaded into active memory https://discordembeds.pc-logix.com/live/1713914306859.PNG https://discordembeds.pc-logix.com/live/1713914307012.png https://discordembeds.pc-logix.com/live/1713914307178.png
L382[23:21:28] <Re​nno> 85.46% memory savings from 1231940 bytes of memory to 179120
L383[23:33:06] <Izzy> so I take it that the idea is to load the recipes into memory as compact strings and read them to get the recipes because strings can use less memory than actual objects?
L384[23:36:11] <Re​nno> pretty much
L385[23:36:18] <Re​nno> and it eliminates the need of opening a file and using a component call
L386[23:36:37] <Re​nno> making the benchmark for retrieval latency rn
L387[23:37:51] <Re​nno> and keep in mind, the LZSS compression is per item recipe in each mod, not per mod, if it were per mod it probably would be higher than 90% memory saving but that might not be a worthwhile tradeoff in retrieval latency
L388[23:42:10] <Re​nno> each individual recipe is unserialized and sorted by the damage (on the end like :11, I refer to it as modifier) with native openos library, then cbor encoded (which is on average ~8% less memory than as a normal table), then after the dictionary of recipes for each item is compiled it gets compressed with LZSS
L389[23:49:41] <walks​anator> trying to run the 1.16.5 fork
<<Prev Next>> Scroll to Top