L1[00:00:24] <Amanda> %remindme 12h it's probably pihole breaking the sleep tracker again, fix it

L2[00:00:24] <MichiBot> I'll tell you "it's probably pihole breaking the sleep tracker again, fix it" in 12h at 07/31/2022 12:00:23 PM

L4[04:22:14] <maumagro> Hello

L8[04:38:16] <yoyou446> peepee

L9[04:38:17] <maumagro> Hello buddy

L10[04:38:21] <yoyou446> D:

L11[04:38:22] <yoyou446> woah

L12[04:38:24] <yoyou446> slay

L13[04:38:29] <maumagro> yeah, and it's online

L14[04:38:33] <yoyou446> can mods see us type on this?

L15[04:38:35] <Ocawes​ome101> not on irc but ima guess those two are from the same ip

L16[04:38:38] <maumagro> nope

L17[04:38:46] <Ocawes​ome101> %hello

L18[04:38:47] <yoyou446> this is so cool

L19[04:38:47] <MichiBot> Hello! Welcome to #oc! The one and only opencomputers channel! Please ask your questions directly (dont ask to ask) and provide error/code examples! (Use pastebin.com if theyre more than one line!) Dont mind the random conversation you might have walked into.

L20[04:38:50] <maumagro> we're on a minecraft server

L21[04:38:56] <Ocawes​ome101> yeah i figured

L22[04:39:00] <maumagro> :)

L23[04:39:02] <yoyou446> hey oca

L24[04:39:11] <yoyou446> ur mother

L25[04:40:14] <Ocawes​ome101> ok

L26[04:41:34] <maumagro> XD

L32[07:19:12] <Va​ur> %tonkout

L33[07:19:13] <MichiBot> Dogast! Va​ur! You beat your own previous record of 13 hours, 8 minutes and 43 seconds (By 2 hours, 56 minutes and 22 seconds)! I hope you're happy!

L34[07:19:14] <MichiBot> Va​ur has tonked out! Tonk has been reset! They gained 0.016 tonk points! plus 0.03 bonus points for consecutive hours! Current score: 3.12196625, Position #1

L40[11:24:01] <Forec​aster> huh

L41[11:24:51] <Forec​aster> I just tried my headphones after gluing them back together (mostly) yesterday and I thought I'd broken the right speaker connection because when testing them there was no sound at all in the right channel

L42[11:25:05] <Forec​aster> I took out the brand new set and it was doing the same thing

L43[11:25:28] <Forec​aster> Playing the video in VLC and telling it to output to the headphones directly works however

L44[11:25:46] <Forec​aster> so for some reason Voicemeeter is not letting the right channel through...

L45[11:26:16] <Forec​aster> or the base

L46[11:30:52] <Forec​aster> wait...

L47[11:33:02] <Forec​aster> huh, the aux input doesn't let any base or right channel through

L48[11:33:03] <Forec​aster> weird

L49[12:00:24] <MichiBot> Amanda REMINDER: it's probably pihole breaking the sleep tracker again, fix it

L50[12:24:50] <Amanda> Fun fact: the mods probably could see what they're typing with a simple Wireshark run

L51[12:25:24] <Amanda> Not that it matters, we've got our own mods

L52[12:28:30] <Forec​aster> wut

L53[13:00:14] <Amanda> Re: the kids earlier thinking they were safe to be rude

L54[13:03:12] <Forec​aster> ah

L55[14:19:05] <Amanda> %choose spaaace or train

L56[14:19:05] <MichiBot> Ama​nda: Some "train" sounds nice

L57[14:30:59] <Forec​aster> but is there space on the train?

L58[14:31:33] <Forec​aster> if not you might need to train the space so it'll go on the train

L59[14:32:41] <Amanda> Oh good, Sweet Transit doesn't work on linux under proton. :(

L60[14:32:45] <Amanda> https://www.protondb.com/app/1612770

L61[14:32:53] <Amanda> Guess I'm waiting for the gaming PC to boot

L62[14:57:50] <Forec​aster> I've seen that, looked neat

L63[14:57:55] <Forec​aster> I've meant to try the demo

L66[16:24:32] <Va​ur> %tonk

L67[16:24:32] <MichiBot> Wut! Va​ur! You beat your own previous record of <0 (By 9 hours, 5 minutes and 19 seconds)! I hope you're happy!

L68[16:24:33] <MichiBot> Vaur's new record is 9 hours, 5 minutes and 19 seconds! No points gained for stealing from yourself. (Lost out on 0.00909)

L69[16:41:17] <Forec​aster> Okay, ammended the script a little, now to apply some updates to the server finally, then reboot, then restart the script

L70[16:41:58] <Forec​aster> assuming none of the changes prevent it from running of course

L73[18:09:42] <Forec​aster> oh, I can't read numbers apparnetly

L74[18:09:53] <Forec​aster> it's only imported 10M systems

L75[18:16:09] <Forec​aster> maybe it'd be easier to dump each system to a file first

L76[18:16:22] <Forec​aster> then process each file into the db afterwards

L77[18:16:37] <Forec​aster> deleting each file as it goes

L78[18:16:53] <Forec​aster> that'd make restarting easier at least

L79[18:17:20] <Amanda> You don't have 200B inodes though, and that'd take up a bunch of space

L80[18:17:31] <Wat​tana> forecaster what u hosting>

L81[18:18:12] <Forec​aster> oh yeah, that thing

L82[18:19:37] <Forec​aster> I could dump 100k systems per file

L83[18:20:20] <Forec​aster> @Wattana what?

L84[18:20:28] <Amanda> I'd say just dump the current index every X ones

L85[18:20:32] <Wat​tana> sounds like ur doing something on a server?

L86[18:20:56] <Wat​tana> or are you just using it to compute stuff instead of your computer?

L87[18:21:58] <Forec​aster> I don't even keep the file position currently but I suppose I could do that easily

L88[18:22:18] <Mic​hiyo> They are loading a list of E:D Systems into SQL

L89[18:22:26] <Mic​hiyo> ALL of the E:D systems

L90[18:22:28] <Forec​aster> I just read the file character by character until I get a complete json object

L91[18:22:48] <Forec​aster> then it parses it, then moves on

L92[18:24:37] <Forec​aster> json isn't ideal for this since you can't easily tell where one object starts and ends

L93[18:24:50] <Forec​aster> would've been easier with xml

L94[18:27:09] <Ar​iri> >Michiyo: ALL of the E:D systems

L95[18:27:10] <Ar​iri> what-

L96[18:34:39] <Mic​hiyo> Star systems in Elite Dangerous?

L97[18:35:11] <Ar​iri> yeah but like what would you need with *all* of that in sql

L98[18:35:39] <Mic​hiyo> Easy queryability?

L99[18:36:04] <Ar​iri> does it have all the system data that everyone has discovered too?

L100[18:36:08] <Forec​aster> Well, for one thing doing this file scan in 50GB every request would be a bit slow

L101[18:36:50] <Forec​aster> depends what you mean by discovered

L102[18:37:04] <Forec​aster> I don't think it has planetary discoveries

L103[18:37:15] <Forec​aster> it does contain literally everything known about a system

L104[18:37:50] <Forec​aster> stars, planets, data on each planet

L105[18:37:55] <Forec​aster> I'm discarding most of it though

L106[18:38:15] <Ar​iri> so what are you doing with it then

L107[18:38:57] <Forec​aster> I mainly want the positions and names, but I'm also storing whether they're inhabited and what class the primary star is

L108[18:39:37] <Forec​aster> hrm

L109[18:39:42] <Amanda> Woo: https://git.camnet.site/drone-images/-/packages/container/packupdate/latest

L110[18:40:04] <Ar​iri> i see

L111[18:40:14] <Forec​aster> I had the script fetch each unique system name into memory before running the import to skip already imported systems

L112[18:40:53] <Forec​aster> this worked fine at the start when I had to restart the script and it had only imported a small amount

L113[18:41:09] <Forec​aster> now when there's 10 million unique names fetching those names is taking a while...

L114[18:41:25] <Forec​aster> I need to change it to query by name first

L115[18:42:04] <Forec​aster> This is the data I'm storing https://tinyurl.com/22o55xxq

L116[18:51:27] <Forec​aster> err

L117[18:51:31] <Forec​aster> this is not ideal

L118[18:51:43] <Forec​aster> I tried just a select query for a name that doesn't exist

L119[18:52:13] <Forec​aster> it's just been sitting there for like 10 minutes now

L122[19:03:56] <Hawk777> Sounds like you might be missing an index or two?

L123[19:05:44] <Forec​aster> index?

L124[19:06:23] <Hawk777> Well, if you’re trying to SELECT WHERE name='foo' and it’s slow, you’d probably want to create an index over name.

L125[19:06:54] <Hawk777> With an index, should be a fraction of a second to query even with millions of rows.

L126[19:09:30] <Forec​aster> the query has been running for almost 30 minutes now

L127[19:12:41] <Hawk777> I imagine there’s a CREATE INDEX command in whatever software this is. Creating the index might take a long time, but once it’s there, queries should be fast.

L128[19:14:24] <Forec​aster> there seems to be a create index statement yeah

L129[19:14:32] <Forec​aster> should probably do that after the import is complete

L130[19:15:27] <Forec​aster> or maybe the index is kept up to date automatically

L131[19:15:35] <Forec​aster> that would make sense

L132[19:39:19] <Hawk777> Once created, adding/removing/modifying rows will also update the index accordingly automatically, as you said.

L133[19:39:29] <Forec​aster> coming up on 1 hour now

L134[19:39:45] <Forec​aster> (still executing the select query)

L135[19:40:03] <Hawk777> That said, it *may* be faster to bulk-insert the data first and then create the index, than to create the index first and then add the data; the latter, since it has to keep the index up to date as each block of data is added, probably involves more shuffling and rewriting.

L136[19:40:52] <Hawk777> I’m very surprised that a select over 10 million rows would take more than an hour, even without any indices, TBH. I’d expect 10 million rows to be scanned in maybe a handful of minutes, not hours.

L138[19:48:47] <Va​ur> %sip

L139[19:48:48] <MichiBot> You drink a sour pussplum potion (New!). Vaur barely manages to catch a green shell that appears in front of them! (Rem. uses: 1)

L140[19:49:48] <Forec​aster> %sip sour pussplum

L141[19:49:48] <MichiBot> You drink a sour pussplum potion. Forecaster barely manages to catch a green shell that appears in front of them! (Rem. uses: 0)

L142[20:21:26] <Forec​aster> I've stopped the query I started in the web interface, running the same one directly on the server now so it can run overnight

L143[20:33:55] <Forec​aster> `mysql> SELECT * FROM `web_edcarriers`.`mcp_systems` WHERE name = 'hams'; Empty set (43 min 11.89 sec)`

L144[20:34:26] <Forec​aster> it ran better directly on the server

L145[20:35:06] <Forec​aster> hrm

L146[20:36:16] <Forec​aster> well... I was about to say it might be better to skip the "does this system already exist" check

L147[20:36:37] <Forec​aster> but then I realized that I can't just import them again since the name isn't the key

L148[20:36:51] <Forec​aster> so I'd still need to find the key by the name

L149[20:37:50] <Forec​aster> so I'll probably just need to let it fetch the existing names anyway

L150[20:37:58] <Forec​aster> biggest problem with that is the memory usage

L151[21:41:28] <Hawk777> Are names unique?

L152[21:44:07] <Forec​aster> Two start systems can't have the same name in E:D I'm pretty sure

L153[21:47:02] <Hawk777> You could put a unique index over the name column. There might be some kind of INSERT OR UPDATE or similar syntax you could use (you’d have to look at the details of whatever database you’re using; I don’t think it’s the same across all of them). I’m also not certain whether that works for uniqueness violations or only primary key violations, but it *might* help.

L154[21:54:01] <Forec​aster> there's `IF EXISTS UPDATE` syntax yeah I know, but that's a lot of extra work

L155[21:54:16] <Forec​aster> I'll just leave it running and see how long it takes

L156[21:54:52] <Forec​aster> it might just run out of memory again, in that case I'll have to do it

L158[22:41:25] <swa​nd45> Hi @Sangar ,

L159[22:41:25] <swa​nd45> I'm the guy that made the GDB stub for sedna and I'm working on adding watchpoints now. I have a question. I see that in `loadSlow` and `storeSlow` TLB entries are only added if the memory device "supports fetch". Why is that? To support watchpoints efficiently, I'd like to add a information to the TLB entries (like I did with breakpoints), but if they aren't always present that approach won't work as well.

L160[23:16:15] <B​ob> damn OC2 is about to bring all the gifted peeps here

L161[23:21:10] <Amanda> %choose laptopnaptime?

L162[23:21:10] <MichiBot> Ama​nda: Once the moon cycle restarts you should definitely do it!

L163[23:21:27] <Amanda> Whats with the obsession with the lunar cycle lately, MichiBot?

L164[23:21:46] <Amanda> Anyway, I'm not going to potentially wait 14 days for that, so laptopnaptime