Stellaris 3.0.3 AI Feedback Megathread

in my classic MP session, we play around 40 to 50 years per session (3h irl) so if someone miss a session, 15 years is not enough. On the otherside, I know now I can start a session with somebody coming a bit later.

I'm writing up a big post on economic AI from a test observer game.

But first, quick thing I noticed in many empires: AIs are typically not upgrading their fleets, even if they have a large reserve of alloys. This seems to be independent of their other economic behavior. Please investigate when / why AIs are not upgrading their fleets.

I made a bug report about the fleet upgrades here (edit: save games are at the bug report and the same as the saves in the post of mine below this one):

3.0.3 (bfcc) does not appear to be significantly improved from 3.0.2 for general AI empire economic AI. Maybe marginally improved. I ran one test game out for 100 years (edit: ok I also did a couple more tests, see later posts). These are almost all the same issues I made 20 bug reports on for version 3.0.2. 3.0.3 beta has not solved critical underlying problems.

Official bug report post is here with screenshots and save games:

Pasting from that post:
I went to observer mode immediately via console commands. "observe" was the only console command used for beginning the test: no fast_forward or anything to do with ticks. I let the game run at fastest speed (minimized window) until the date specified in the furthest dated save file name, saving along the way. I then began inspecting what the AI did with its empires via the "play" command to take them over and see warnings/planets/etc (reloading older saves). I will post discovered problem empires one by one. All default settings (as before, including difficulty, thus no AI bonuses) and only selected empire was United Nations of Earth.

The attached screenshots detail particular problematic empires. The naming convention for the PNG screenshots is (empire #)-(year).png.
For example, 7-2300.png is the empire via "play 7" at year 2300, so reference the 2300.12.02.sav file.

The problems:
0-2252, lacks scientists
0-2300, has max influence not being used on edicts, all planets with stability and/or housing/employment problems
2-2300, has max influence not being used on edicts, lacks scientists, out of food, stability problems on many planets
3-2225, out of minerals, out of consumer goods
3-2252, 27 years later still out of minerals, out of consumer goods, planets losing stability
3-2300, same story as past 75 years
4-2225, many habitable planets nearby uncolonized with no colony ships built either
4-2300, max minerals not being sold off, stability problems on many planets
6-2300, fail cascade, 1 planet and no energy, no food, no consumer goods, max influence not being used, stability problems
7-2300, max minerals not being sold off
8-2300, out of food, max influence not being used, stability problems

(note that I did not inspect the 2266 and 2287 save games but am just providing them to help dev investigation)

For AI feedback megathread convenience I'm attaching the files here, too. No mods, all dlc/expansions except nemesis and necroids.

ALSO: if any other player is reading this, please load up the saves and feel free to figure stuff out. I only looked for the most obvious problems and I did not look for root causes.

One thing I liked is that the AI in my games were willing to go to war much earlier than I have seen in the past. Yes it was a devouring swarm and I was set to aggressive AI, but I'm glad it didn't just let me get away with no fleet to speak of. Maybe it was a fluke, but if it wasn't good work! I'd still like to see "Hostile" militaristic empires shutting their borders down faster and pushing that relationship malus more aggressively.

The only issue is that because Fighter Bastions are so overwhelmingly powerful, the AI was unable to do anything but stare at me the entire war. I really think fighters should be a tier 2 tech because they allow you to just completely shut down early aggression for a fairly trivial cost. 300 alloy Bastion completely shut down thousands of alloy worth of enemy fleets. I get that bastions should be more economical because they are stationary, but that's a bit much.

I think the goal here should be to make the hostile AI empires (and space fauna) aggressive enough so that players feel the need to devote resources toward defense early on. If you don't push the player on this, they're just going to tech-rush + rapid expand while the AI gets left behind because they had a more balanced build. Something I would really like to see is getting the Space Fauna more involved in the early game when their small fleets are still relevant. Having them pressure neighboring outposts (and actually destroy them instead of merely disabling) would create some low-stake early game combat that I think would be good for the game.

I am getting crash to desktop while launching the game with 3.0.3 on Linux (two different computers). Could not even get to main menu.

pmchem said:

ALSO: if any other player is reading this, please load up the saves and feel free to figure stuff out. I only looked for the most obvious problems and I did not look for root causes.

Click to expand...

This is really in the purview of Paradox's QA job, but I took a quick look at your save and I see commonalities between our saves, food and amenities are worrisome low across some empires and if the AI is blocked from building Holo-Theaters and Clerks are a lesser valued job and are now in fewer numbers with the latest balancing (only trade value was doubled), then I'm not sure which black hole the AI is supposed to pull its Amenities from if they aren't Religious and have Temples or a specific civic that alters the setup of the planetary capital.

Fundamentally 3.0.1, 3.0.2 and 3.0.3A and 3.0.3B releases are the same, there's a core issue that overshadows any other feedback and before they are addressed there's not much point commenting on beating the AI in tech or what not.

I ran 30 years earlier in observe and posted it in the bug report forum for reference. https://forum.paradoxplaza.com/foru...-economical-collapse-within-30-years.1472749/

An AI issue I noticed is that one of the empires in my game with the post-apocalyptic origin is not settling any worlds.



This is causing them to get left behind dramatically. He has tomb world and desert preference with desert planets nearby and he also has +20 habitability trait. Save attached.

Edit: 12 years later they FINALLY dipped their toe into expanding to the 2 desert planets. However, by now, they are very far behind.

The changes to Clerks seem good so far, however it seems the update hasn't touched Machine Empire's Maintenance drones. It is the same problem Clerks had but worse, as Nexus Districts provide 3 Maintenance Drone jobs each rather than 1 Clerk as Normal Empires' Housing Districts do.

As Maintenance Drone jobs have a high priority, you end up with Drones working on those and leaving Energy/Mineral production slots completely empty.

I think there should be less Maintenace Drone jobs but have each produce more Amenities.

I observed a habitat dweller AI crippling itself with the stop population growth buttons because one of its habitats got full. This is a complete waste now, as full planets are still helpful due to auto-migration. Worse, they're using the "stop assembly" button without even having a robot assembler -- if the AI is going to push that button at all, it should just disable or delete the assembly plant instead and avoid the associated extra penalty.

Looking at the code, the AI is using the "Stop Robot/Droid Assembly" and "Discourage Growth" decisions when it is out of housing and finished building districts:

IMO these two decisions should just be removed from the game, or at least prohibit the AI from using them as they don't make any sense in the new pop growth system.


I'm curious if the AI can properly play a void dweller civ in the new system, but when it shoots itself in the foot early through these buttons there's really no way to tell.

As for more systemic issues, I suspect one big problem with the AI right now is it is doing the wrong things when under occupation. I reported a bug about this

The AI does two very wrong things when at war: it looks at the (temporary) loss from devastation and thinks it has a critical housing shortage, and it tries to prioritize building on enemy-occupied planets. I suspect the latter might result in the AI just getting stuck and doing nothing for a good chunk of the war, if all the "important" projects are impossible.

Normally the AI tries not to build things unless it has workers to take the jobs, but this seems to be broken when the AI has chattel slaves. Looking at some slaver empires in a midgame save there are a bunch of planets full of buildings that have unemployed slave workers and almost fully empty specialist jobs.

This problem only gets worse as the empire gets more and more different kinds of xeno slaves, as the base species becomes increasingly less likely to grow (and the unemployed slaves won't auto-migrate either!)

Looking under the hood, the AI seems to decide what slavery/citizenship policy to use mostly randomly based on their government type rather than what makes strategic sense. If there's a planet full of empty specialist jobs the AI should strongly consider promoting the slavery type to indentured for the dominant slave pop on that planet. Similar logic should apply to freeing a species from slave to residence status when there's too many unused ruler jobs and no free empire pops have suitable habitability.

Note that indentured servants are affected by this bug which prevents them from becoming entertainers (not really noticed since the AI doesn't appear to make entertainers, a different serious fault)

This is a bit of a peanut gallery post but is the AI now aware of industry/forge world designations affecting the distribution of artisan/metallurgist jobs? It seems a common "death spiral" triggers when the AI wants consumer goods/alloys, and it attempts to build industrial districts on a planet with one of those designations. If it's the wrong designation (I.E. it wants consumer goods but builds on a planet with a forge world designation) it'll get the wrong resource and often bankrupt itself.

Played about 100 years tonight. On Ensign difficulty, there's less rebellions from what I can see. Virtually every other empire in my contact list is listed as pathetic in power. I got lucky (or unlucky) that I spawned in a very isolated part of the galaxy, I didn't meet an AI empire until like 2260, so unfortunately I don't have any insight into how well the AI did in the early game.

Some of the past ethics rebellions I had in the past suspected were due to bad planetary management, might actually be due to a quirk in the faction system where empire rulers can join factions outside of their governing ethics, which in turn attracts pops into those factions, tanking happiness and then stability. I came to this conclusion because my militarist, spiritualist, authoritarian dictatorship had its first dictator lead the xenophile faction for some reason. This was an annoyance for a player, and I can easily imagine how an AI might fail to recover from that weird situation

Edit Update 1: Played more today. My assessment of the AI based on my neighbors I can see is that in some cases it seems like the AI just isn't settling enough planets. However, I just had a war with my one neighbor that is settling planets at a decent clip, but they are so far behind me on tech and fleet size that I was able to walk all over them.

Also, this is feedback on pop growth and economy, but it feels like it's in a pretty good spot now. The changes to buildings makes the feeling of available jobs relative to population feel much better.

Edit Update 2: It's hard to tell whether the slave processing facility is actually resettling pops. Here is what is happening in my game: I am abducting pops who wind up on my capital. They exceed the number of jobs I have for workers, so they become unemployed which means they are supposed to be resettled eventually. But I am also noticing that pops are winding up on the slave market. What I want to happen: I want those unemployed slaves to not be sold, I want them to quickly get redistributed across my empire. Maybe there should be a higher tier of slave processing facility for empire capitals that auto-distributes them to open jobs?

Also, play experience around the year 2390 is that the AI isn't really progressing into the late game in a strong way. I am on repeatable techs and capped on storage for most resources. I am not expecting the AI to be done with the tech tree or anything, but it seems like at least some of them should be building titans and fleets of battleships by now.

Well I see a bit of a problem...there's an empire with a system rebelling 37 years in. Turned out after a bit of playing, the problem was Doomsday origin. The AI just...let itself die, basically.
If Doomsday can't be fixed with the AI (I'd think you could tell them to stop developing the homeworld and focus colonization, then maybe add a thing so the AI knows what resettlement is...), please disable it for random AI empires. It utterly breaks them right now.


Aside of that bit of garbage, both the empires next to me that tried to conquer me did the usual boneheaded AI move of moving in one fleet at a time and fighting under a starbase even before FTL inhibitors are a thing, and not preparing their fleets to be in position before a war. They seem to be managing internal affairs okay, at least.

EDIT: More rebels...
It's quite clear that there are still serious problems in terms of AIs splitting apart.


EDIT 2: Wow, okay, so I guess the AI doesn't understand negative value... Do note, at some point these guys killed the OTHER rebel empire in the same system, no idea how that one worked to begin with.

Overall first impression (on Ensign difficulty):

-AI does not implode into ten one-system rebel empires. This has improved (well, relative to 3.0.2) a lot, good.
-AI builds some fleets, and is reasonably good at keeping up with economy until about 2280. This is better than before, but could use more work, I suspect this has to do with how the AI doesn't attempt to get rid of clerks in favor of more productive jobs if it can
-AI does not tech-rush even if it has the opportunity, out-teching the AI is as easy as ever (nor does it seek to punish a tech-rushing neighbor who has neglected their fleet - this might be in part due to how good hangar border stations are)
-AI does not grow their planets well, nor plan habitats well (in terms of jobs and placement, respectively)
-AI splits their fleets up too much (from what little I could see). This is effective in carpet-"sieging", but should only be done with a portion of the overall force, and certainly not when fighting an equally-strong opponent. This has a place when fighting weak foes, or hit-and-run skirmishes when fighting a stronger one (though smaller splits are probably preferable in the second case). Definitely a bad idea when fighting the Crisis (or Khan, or Tempest).

... If I as a player, playing peacefully, do not expand past 5 planets at the start because I've been boxed in and still manage to out-economy AIs with double that planet count, something is seriously off. I'll be running observer games next week to take a peek at what the AI is doing.

EDIT 1: So I let a game run to 2256 in observer mode, there was a period where an advanced start UNE was having economic issues and borderline had rebels due to a persisting food shortage, but they seem to have sorted it out. What is worrisome, however, is how little priority the AI seems to have on technology. It's 2256, and yet not a single AI built a single research lab. Yep, that's right, all of them are sitting on their one starting lab plus whatever space research they have. Not to mention placing alloy foundries and civilian industries when building another industrial district would've done the job just as well. There is zero effort in specializing planets in one kind of resource production or another, and clerks are eating up valuable pops that could be doing better jobs - the AI does not consider clerks to be "unemployed" and as such doesn't look to expand job opportunities based on that. Not even touching on how "logistic growth" puts the AI further behind, but that's ok for now, since that's not a fixed system yet.

In short: AI does not put any priority on improving research. AI fails to realize economic potential of their population due to clerks. AI does not adjust their development plan based on their cosmopolitical situation (*).

EDIT 2: Ok I have to take it back a bit, there are still plenty of rebels, just not in absolutely every AI state anymore. After looking around a bit I'm not even entirely sure what's causing all the rebels, sometimes it's a lack of resources leading to happiness tanking and thus rebels, sometimes it's amenities being at like -21 on a fully-staffed planet.

The very first thing an advanced start AI did on its second planet, with 3 available building slots, was to build a city district.


This was despite having enough resources to go in just about any direction:



Meanwhile its prioritized goal (to build an alloy foundry on its capital) is clearly suboptimal, as its capital is already full, cannot fit more industrial districts and the planet has an industry building already. Its decision to replace a bunch of clerks with a holotheater seems reasonable, however I suspect it won't actually replace the clerks and will instead just leave them in place forever:



I cannot think of any reason why building that city district would be a good idea compared with basically every other option: research lab, industry district, alloy building, etc.

The AI must be highly valuing either extra housing or an additional empty building slot when it already has plenty to work with.

MrFreake_PDX said:

Hello Community!

Please be sure to read this post in its entirety before responding to this feedback thread. Posts that do not follow the rules will be removed.

As you know, we’re always looking for ways to improve the Stellaris AI, as such we’ve made some further changes to the 3.0.3 Open Beta. In addition to further fixes and balance changes, this version includes several AI fixes to better manage the lower pop numbers in 3.0.

Here are the changes in the current updated 3.0.3, for the full patch notes see this post.

Spoiler: Updated Steam 3.0.3 Open Beta Changes

#################################################################
######################### VERSION 3.0.3 ###########################
#################################################################

###################
# Balance
###################

* Reduced the number of Clerk jobs provided by buildings and districts by 40%.
* Clerk trade value has been increased to 4.
* Crime Lord Deal now also adds criminal jobs.
* Manufacturing focus buildings (factories and foundries) are no longer exclusive from one another, and no longer add jobs to Industrial Districts. They instead increase the base production of alloy or consumer goods producing jobs by 1 or 2, with a corresponding increase in upkeep.
* Buildings that increased basic resource production and added jobs to basic resource producing buildings or districts (Energy Grids, Mineral Purification Plants, etc.) now increase the base production of the relevant jobs by 1 or 2 based on tier instead of their previous modifiers. Machine empires still gain the extra resource district slots as before. Yes, things like "livestock" counts as a "relevant job".
* Reduced base automatic resettlement chance to 5% per month (46% chance per planet to resettle per year).
* Reduced logistic growth ceiling from 2.0 to 1.5.
* Sliders have been added in Galaxy Generation for the planetary Logistic Growth cap and Empire-wide Growth Required scaling values. Please note that adjusting these can have significant effects on game balance and performance.

###################
# AI
###################

* AI now cares more about energy & alloys. Added a building limit define.
* AI now waits 10 years to fully take over players.
* Improved AI human takeover behavior
Code:
** bWasHumanRecently added to weighting stuff in CalcBuildingWeight
** "HandlePlanets" now checks for AI_FREE_JOBS_BUILDING_BUILD_CAP in order to build misc buildings
** Removed the Upkeep check for approximation because of locking itself out of building certain stuff in deficit
** AI stops building armies after takeover
** AI stops building starbases & starbase modules after takeover
** AI stops buying & selling pops after takeover
** Added some comments to pop.h and pop.cpp

Script
** Reduced amount of jobs allowed to build new stuff
** Increased scores for deficit, focus & amnetie building weights
** Reduced weight for pop buildings
** Increased AI takeover timer to 10 years
** Updated economic plans to not change as much through the game time
** Economic plans now favor less research and more stability and economic balance
** Added a job weight for low income for artisan jobs
** Increased job weight for technicians for low income situations
** Reduced low income threshold for miner jobs
* Economic plan fix for hive & gestalt

###################
# Modding
###################

* LOGISTIC_POP_GROWTH_CEILING, REQUIRED_POP_GROWTH_SCALE, and REQUIRED_POP_ASSEMBLY_SCALE defines have been removed as they have been replaced by sliders in galaxy generation.

###################
# Bugfixes
###################

* Updated numbers in espionage operation roll tooltip to show correct information
* Fixed a too-long string cutting off invader power values in ground combat view in French
* Fixed an issue where destroying the Contingency's final world with a Star-Eater would not not properly end the Contingency Crisis
* Further fixes to the end of the Cybrex precursor chain
* The tooltip when you can't reinforce a fleet shouldn't duplicate the cause.
* Fleets can be reinforced even if one ship type in the fleet can't be built.
* Fixed a crash upon mousing over a system with planets in as an observer
* Fixed a couple of Edicts having wrong deactivation cost
* Thought Enforcement tech is now researchable with Utopia.
* Fixed an issue where Nomads would keep asking for a Planet over and over, before establishing communications
* You can no longer invite someone you are at war against to join your side in a new war

These changes give the AI a greater focus on economic stability and improves some research-related behaviors, but are also a work in progress and will continue to be updated in future patches.

We will continue to fix some more issues internally and update the beta branch as needed until we are happy with a final product that we can release sometime at a later date. This Open Beta will only be available on Steam.

Once again, we are looking for directed, constructive feedback on the current AI as it stands in 3.0.3 (bfcc) Beta without mods. There is a thread for open discussion here.

What we are looking for:

• Feedback on your playthrough from the 3.0.3 [bfcc] Open Beta, specifically regarding how the AI performs economically.
• How does the AI keep up with the player, economically on “Ensign” difficulty?
• Is it easier to defeat an AI empire than before the changes?
• Do the AI empires break apart more or less often than before?
• Is it too easy to out-tech the AI now?
• If an AI takes over for a player in an MP game it should stay fairly inactive for about 15 years (no wrecking the economy, no spending ascension perks, etc…) - how does that feel? Too short? Too long? Is there still something the AI does that it shouldn’t in this case?

Not Useful replies

• Do not reply to other users. Reply to other users in the discussion thread here. Encourage them to change their mind and edit their post, in the discussion thread. Be nice with your replies in the discussion thread. Did I mention the discussion thread?
• It is important to note that any mods that add anything to the game that has an AI_weight can affect the changes included in this version. This includes planet, building, ship and AI mods. Please disable all mods before testing.
• Your perception of the changes, without actually playing the open beta
• Do not use the fast_forward command for testing, as it produces unreliable results (compared to letting the game run in observer mode)
• Other game issues not related to the AI’s behaviour
• Posts attacking the Stellaris team members
• Off topic/Not applicable posts will be removed.
• Feedback from playthroughs using checksum changing mods. (Ie. UI/graphics mods are fine)

To opt-in to the Steam open beta branch, right-click Stellaris, click Properties, Beta tab, and choose “Stellaris_test” from the drop-down.

Please only reply to this thread once, with your feedback on AI behavior in the updated 3.0.3 open beta.

Click to expand...

Played almost fulltime on 3.0.3beta & heavily after the Clerks/Crime Lord Criminal spawn update came out.

So now I have noticed from Megacorp looking at branch offices and getting Intel, that the AI empires are sacrificing long term development for heavily developing basic resources and manufacturing ie NOT ENOUGH RESEARCH. I am playing on a huge galaxy scale with 2 habitable planets and used Admiral with scaling, crisis 1.5x (later game needs challenge too), so apart from first colony (and then second) means you probably should tech up and turtle a little (because empires are far apart), as then after saving influence and spending on resources, you are more efficient. Building star bases, when systems don't have many resources leaks energy.

My coop pacifist, materialist, egalitarian federation partner, is doing OK, had a lot more fp than me (as usual) but very low tech development (1/3 my tech level), despite being given research agreement by me as part of striking an early commercial pact. This partner is doing quite well out of this as I developed branch offices with merchants to boost trade and add amenities, I am in mid-early-game about 2249 and have 3 offices, one planet has over 50 pops but it didn't upgrade the admin, so I can't add another branch building; it's only the original research lab, plus industry districts and basic resources. Another is set to bureaucracy but has a fair amount of trade from some clerks. Other materialists are also way behind me tech wise, though there's no community yet; that takes a good while to form on huge.

In my own empire enabling auto management, I started off a small mining world, it has robot assembly, mineral +1 building and a district with miners and another waiting for pops/droids. The AI is building, building, building so is wasting energy and reducing planet capacity for no reason. The AI should be great at calculating when to build things just in time.

The other AIs are also miles behind me on tech, I had a strange early contact from one far across the galaxy, which apparently tasted nice according to the gourmet's notes, before that ship ironically became a meal. Anyway it was also a materialist, being so far away I hoped the AI would want a research agreement, but no it's sticking to it's belligerence despite me being very far away. I tried everything, but gave up and went for the cheapo rivalry to earn influence.

Another empire I am in contact with is Rogue Servitors with Lithoid Trophies, based on Stefan Anon's Remnant Build. I have struggled to gain much intel on them, ran opos and have basics except fleet power, but it's clear they also have fallen way behind tech wise, probably were over keen on building a fleet without any threat. Then it decided after a while to treat me as a threat despite being an Xphile/Fan Mat. Megacorp, rather than look for a research deal; it started harming & rivalled me, when our bases hadn't quite reached contact; despite my peaceful overtures.

Now in my previous run, which was 3.0.3beta1 but became 3.0.3beta2, when the empires met up at Galactic Community, my fairly coop empire designs had done pretty well, I've added them into games as I was really sick of only moronic xphobe / militaristic if I play as UNE or mostly Megacorps (who are rivals) when a Megacorp. When I tried a technocratic build, based on Stefan Anon's recommendations (except I passed on slavery bit), I had an aggro bullying empire, surrounded by 3 materialists, one first contact with the Doomsday Origin which must have began with Diplomacy as it invited me to a Research Coop v. early game.

We need a mix of empires, the cooperative ones should seek to meet up and share contact info, like I did in this game, so we could contain any genocidals. As it stands, Dick 3.0 appeared set up for Xphobe/Fan militarist like the Commenwealth of Man, I tried them and pre beta2 clerk changes, the game fitted so much better. Now I bought the Necroid & Nemesis DLC although there's no appeal in being a necrophage for me, the crisis or needing the validation of the custodian because I wanted to understand the intel stuff to compare with/without DLC in explanatory videos.

Funnily enough, I need to become a reformed character as CoM because the xphile AI militarist, despite my coop and fighting same threatening oponent, decided I was the threat and started guaranteeing the independence of my first victim, whose homeworld I had taken and avoided worse in 2 wars by timely concessions, as I had saved up the influence to increase my claims. In that case, I don't understand how that empire so quickly gained the intel to decide it should turn on me. It's probably a wise decision, but UNE soon turned cool too rather than team up on the xeno sandwhiched between us.

The early AI seemed stronger than in 3.0.2. In my 3.0.3 game that I played, a neighboring AI fanatic purifiers steamrolled me with corvettes. I will try to play better next game.

Update: I started another game. This time I played better. It is 2274, and I am significantly stronger than any regular AI empire. Though I am not a hegemon.

in case my initial test was not representative of 3.0.3 (edit: 3.0.3, it's all 3.0.3 beta), I decided to let a few more tests run and see if anything unusual popped up. This post reports on a test that ran overnight to year 2417 using my usual test procedure (observer mode, default settings, no mods, etc.):

screenshots and save games also uploaded at bug report

The attached screenshots detail particular problematic empires. The naming convention for the PNG screenshots is (empire #)-(year).png.
For example, 9-2417.png is the empire via "play 9" at year 2417.

0-2417: inactive civics, serious crime and unemployment problems on empire capital, which also has many free building slots
2-2417: huge unemployment problems on several planets with no apparent efforts to resolve them
8-2417: low stability, including a zero stability world with 3 pop but... 28 free housing slots?
9-2417: huge unemployment problems on several planets with no apparent efforts to resolve them

and of course the typical "not upgrading fleets" and "max influence not being used" problems for various empires

EDIT: I just realized the OP asks people to only post once. my bad. that seems hard to make work with ongoing testing, though?? anyway, I'm done with tests for now.