Commit Graph

33 Commits

Author SHA1 Message Date
Spottedleaf c300745c8b even even even even more work 2020-06-25 16:38:24 -07:00
Aikar 6c87b85dd1
1.16.1 prep 2020-06-24 22:00:02 -04:00
Aikar ce270e1412
Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
b2f1908c SPIGOT-5783: Add helpful info to UnknownDependencyException
e4f46260 SPIGOT-2623: Add EntityEquipment methods to get/set ItemStacks by slot.
529a9a69 SPIGOT-5751: Clarify behaviour of block drop-related API methods

CraftBukkit Changes:
8ea9b138 Remove outdated build delay.
ffc2b251 Revert "#675: Fix redirected CommandNodes sometimes not being properly redirected"
cb701f6b #675: Fix redirected CommandNodes sometimes not being properly redirected
c9d7c16b SPIGOT-2623: Add EntityEquipment methods to get/set ItemStacks by slot.
fad2494a #673: Fix Craftworld#isChunkLoaded
8637ec00 SPIGOT-5751: Made breakNaturally and getDrops returns the correct item if no argument is given

Spigot Changes:
a99063f7 Rebuild patches

Fixes #3602
2020-06-23 04:40:03 -04:00
stonar96 de6dfedfef
Make Anti-Xray multithreaded (#3520)
Obfuscate multiple chunks at a time over the server thread pool.

Will speed up chunk processing when anti xray is enabled.

Co-authored-by: Aikar <aikar@aikar.co>
2020-06-09 04:12:20 -04:00
Aikar b2d81e21c5
Improve Chunk Prioritization and Internal Scheduler
In previous MC versions, we had a rather simple internal scheduler
for delayed tasks that would just keep pushing task back until desired
tick was reached.

The method it called to schedule the task changed behavior in 1.14, and now
this scheduler is not working nowhere near what it was supposed to be doing.

This was causing long delayed task to eat up CPU (In Oversleep for example)

Rewrite this to just use the CraftScheduler for scheduling delayed tasks.

Once this was fixed, it became quite clear the code that delayed ticket
additions for chunks based on distance was clearly not right, as it was
tested on the previous broken logic.

So the ticket delay process has been vastly revamped to be even smarter.
Chunks behind the player can load slower than the chunks in front of the player.
We also can delay ticket adding until one of its neighbors has loaded, as
this lets us get a smoother spiral out for the chunks (minus frustum intent).

Additionally on frustum previous commit inadvertently broke frustum trying to
fix an issue when the real fix lied elsewhere, so restore chunk priority so
it works again.
2020-06-09 03:17:25 -04:00
Aikar 3d2bc848c1
Ensure VillagerTrades doesn't load async - fixes #3495
In rare cases, this class could potentially be loaded from
the chunk threads causing it to initialize async and cause errors.

This would then break the server and chunk saving.

So ensure its loaded at start of server to avoid this.
2020-06-03 00:00:17 -04:00
Aikar f4a47db699
Improve Thread Pool usage to allow single threads for single cpu servers
Switch to a standard fixed size ThreadPoolExecutor as we don't use the
advanced capabilities of a ForkJoinPool.

ForkJoinPool does not allow single threads, and really rather not use
2 different executor types based on core count.

Also, change thread priorities so that main thread is prioritized by
the OS at a higher priority than the other threads. May not help too much
but it at least signals the OS the information to know main is more important.
2020-06-02 02:19:07 -04:00
Aikar 70ad51a80c
Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

My recent work on serialization is now in CraftBukkit so was able to drop the patch and Paper
is now consistent with upstream.

Bukkit Changes:
e2699636 Move API notes to more obvious location

CraftBukkit Changes:
1b2830a3 SPIGOT-4441: Fix serializing Components to and from Legacy
2020-06-02 01:29:18 -04:00
Spottedleaf 0d5ec65372
Cleanup async chunks (#3456)
Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
2020-05-28 19:34:58 -04:00
Aikar c9795e9221
Fix Spigot bug with chunk unloading
Spigot inserted their Slack Activity Accountant in the wrong location
resulting in a chunk being removed from the unload queue, inserted into
the unload map, but never calling the function to finish the removal....

This caused the chunk to become stuck in the unload map if ever hit, because
the unload map was meant to be a TEMPORARY location while it was saving.

Fix this by abort iteration AFTER the current chunk is finisehd processing

Also, improve how aggressive we are at unloading chunks, targetting 10% per tick instead.
These saves are asynchronous so there should be less of a hit here.
2020-05-25 11:09:44 -04:00
Aikar 8b2a437b9e
Fix plugins calling getChunkAtAsync asynchronously
While this method has async in it's name, it's not actually meant
to be called asynchronously.... It just means IT will load the chunk
asynchronously without blocking main.

So fix this so that if a plugin calls it async, it forces the request back to main thread.
2020-05-23 18:38:29 -04:00
Aikar 7c001d64a5
More Improvements to Chunks
Fixed issues where urgent and prioritized chunks didn't actually
always get their priority boosted correctly....

Properly deprioritize non ticking chunks.
Limit recursion on watchdog prints to stop flooding as much
Remove neighbor priorities from watchdog to reduce information
reduce synchronization duration so that watch dog won't block main should main actually wake up
probably fixed a deadlock risk in watchdog printing also that was leading to crashes
fixed chunk holder enqueues not being processed correctly
added async catchers in some locations that should not be ran async

Fixed upstream bug where VITAL callbacks that must run on main actually could
sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately!

This build will provide massive improvements to stability as well as even faster
sync chunk load/gens now that priority is correctly set.

Fixes #3435
2020-05-22 23:52:56 -04:00
Aikar 4d38ee111f
Many fixes and improvements to chunk prioritization
I believe this brings us back to stable. A lot of complexity was
learned about juggling priorities.

We were essentially promoting more chunks to urgent than really
needed to be urgent.

So this commit adds a lot more logic to juggle neighbor priorities
and demote their priority once they meet the requirements needed of
them.

This greatly improves the performance of "urgent" chunks".

Fixes #3410
Fixes #3426
Fixes #3425
Fixes #3416
2020-05-22 01:03:42 -04:00
Aikar 7e1525ea2d
Many improvements to chunk prioritization and bug fixes
Fixed a few bugs, and made numerous improvements.
Fixed issue where a sync chunk load could have its ticket removed and the
priority ticket could expire...
Still not perfect there but better than before.

Also fixed few other misc issues such as watchdog cpu usage, chunk queue update
had risk of double enqueue due to it no longer being a set.

Added much more information about chunk state to watchdog prints.

I see some more room for improvement even, but this is much better than before.

Fixes #3407
Fixes #3411
Fixes #3395
Fixes #3389
2020-05-20 05:13:01 -04:00
Aikar 614a664bd3
Implement Chunk Priority / Urgency System for Chunks
Mark chunks that are blocking main thread for world generation as urgent

Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.

Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.

This massively reduces the lag spikes from sync chunk gens.

Then we further prioritize loading order so nearby chunks have higher
priority than distant chunks, reducing the pressure a high no tick
view distance holds on you.

Chunks in front of the player have higher priority, to help with
fast traveling players keep up with their movement.

This commit also improves single core cpu scenarios in that we will
now automatically disable Async Chunks as well as Minecrafts thread
pool.

It is never recommended to use async chunks on a single CPU as context
switching will be slower than just running it all on main.

This also bumps the number of server worker threads by default too.
Mojang does not utilize the workers in an effecient manner, resulting
in them using barely any sustained CPU.

So give it more workers so more chunks can be processed concurrently

This change also improves urgent chunk loading, so players flying into
unloaded chunks will hurt a little bit less (but still hurt)

Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 04:09:37 -04:00
Aikar 897dd2c840
Foundational work for Future Memory usage improvements
This commit doesn't do much on its own, but adds a new Java Cleaner API
that lets us hook into Garbage Collector events to reclaim pooled objects and
return them to the pool.

Adds framework for Network Packets to know when a packet has finished dispatching
to get an idea when a packet is done sending to players.

Rewrites PooledObjects impl to properly respect max pool size and remove
almost all risk of contention.

Bumps the Paper Async Task Queue to use 2 threads, and properly shuts it down on shutdown.
2020-05-16 21:38:19 -04:00
Aikar 4ae089597b
Fix collision checks on spawning hanging entities and null on async chunk loads
getCubes will now always load chunks
getChunk with gen false will now not throw error

Fixes #3368
Fixes #3364
2020-05-14 05:05:13 -04:00
Aikar 827cc63263
Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
ffc8e4ca SPIGOT-5716: Clarify documentation of MultipleFacing

CraftBukkit Changes:
d07a78b1 SPIGOT-5716: Clarify documentation of MultipleFacing
46a13860 SPIGOT-5718: Block.BreakBlockNaturally does not reflect tool used
214ffea9 SPIGOT-5727: GameRule doImmediateRespawn cannot be set per-world

Spigot Changes:
2f5d615f SPIGOT-5730: Modernise inventory patch
a2bdb119 SPIGOT-5679: Add config option for end portal activation sound

Closes #3352
2020-05-12 01:27:07 -04:00
Aikar 8e9459eadc
Fix missing flag pass for isUrgent 2020-05-11 00:19:40 -04:00
Aikar a054aa6f02
Properly remove Entities from current chunk
We store a reference to the chunk the entity is currently in, so use it
to more accurately unregister it in chunkCheck

Should maybe fix some entity loss issues.
2020-05-10 05:52:31 -04:00
Aikar e5f6489602
Add Urgent API for Async Chunks API and use it for Async Teleport
This also cleans up the implementation of Async Chunks to get rid of most
Consumer callbacks and instead return futures.

This lets us propogate errors correctly up the future chain
(barring one isn't lost even deeper in the chain...)

So exceptions can now bubble to plugins using getChunkAtAsync
2020-05-10 00:57:03 -04:00
Aikar f6fc0d4cd4
Don't create region files for chunk checks - mistake on upstream merge
Accidently handled a conflict wrong in a previous upstream merge and set wrong param.
2020-05-08 18:22:37 -04:00
Aikar 36f34f01c0
Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
da9ef3c5 #496: Add methods to get/set ItemStacks in EquipmentSlots
3abebc9f #492: Let Tameable extend Animals rather than Entity
941111a0 #495: Expose ItemStack and hand used in PlayerShearEntityEvent
4fe19cae #494: InventoryView - Add missing Brewing FUEL_TIME

CraftBukkit Changes:
933e9094 #664: Add methods to get/set ItemStacks in EquipmentSlots
18722312 #662: Expose ItemStack and hand used in PlayerShearEntityEvent
2020-05-06 06:05:22 -04:00
Aikar f37381ea8a
Optimize Network Manager to not need synchronization
Removes synchronization from sending packets
Makes normal packet sends no longer need to be wrapped and queued like it use to work.
Adds more packet queue immunities on top of keep alive to let the following scenarios go out
without delay:
  - Keep Alive
  - Chat
  - Kick
  - All of the packets during the Player Joined World event

Hoping that latter one helps join timeout issues more too for slow connections.

Removes processing packet queue off of main thread
  - for the few cases where it is allowed, order is not necessary nor
    should it even be happening concurrently in first place (handshaking/login/status)

Ensures packets sent asynchronously are dispatched on main thread

This helps ensure safety for ProtocolLib as packet listeners
are commonly accessing world state. This will allow you to schedule
a packet to be sent async, but itll be dispatched sync for packet
listeners to process.

This should solve some deadlock risks

This may provide a decent performance improvement because thread synchronization incurs a cache reset
so by avoiding ever entering a synchronized block, we get to avoid that, and packet sending is a really
hot activity.
2020-05-06 05:28:47 -04:00
stonar96 8f9df2ed43
Anti Xray cleanup
Undo the accidental renaming of a method in 0aad8bf
Aikar wanted to rename DataPalette#getDataBits(T object) to getOrCreateIdFor
in 0aad8bf but he also accidentally renamed
ChunkPacketInfo#getDataBitsIndex(int chunkSectionIndex) to
getOrCreateIdForIndex.

Remove chunk-edge-mode and chunk loading entirely from Anti-Xray
The chunk-edge-mode is broken since several versions.
Loading chunk neighbors for chunk edge obfuscation isn't needed anymore.
Unlike in previous versions, these are under normal circumstances already loaded
at the time we need them (plugins for example can bypass this).

Use the modified methods and constructors everywhere
Anti-Xray provides support for the default nms methods and constructors,
which where modified by Anti-Xray to avoid breaking stuff (plugins)
which somehow uses these methods.
However, the modified versions of those methods and constructors should be used
where possible.
2020-05-06 04:35:20 -04:00
Spottedleaf b4e629a283
Use distance map to optimise entity tracker / Misc Utils
Use the distance map to find candidate players for tracking.

This also ports a few utility changes from Tuinity
2020-05-06 03:47:24 -04:00
Aikar 57dd397155
Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
b999860d SPIGOT-2304: Add LootGenerateEvent

CraftBukkit Changes:
77fd87e4 SPIGOT-2304: Implement LootGenerateEvent
a1a705ee SPIGOT-5566: Doused campfires & fires should call EntityChangeBlockEvent
41712edd SPIGOT-5707: PersistentDataHolder not Persistent on API dropped Item
2020-05-01 18:03:57 -04:00
Aikar 842e040c19
Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
220bc594 #486: Add method to get player's attack cooldown
21853d39 SPIGOT-5681: Increase max plugin channel size
5b972adc Improve build process
b55e58d9 Note which custom generator is missing required method

CraftBukkit Changes:
893ad93b #650: Add method to get player's attack cooldown
ef706b06 #655: Added support for the VM tag jansi.passthrough when processing messages sent to a ColouredConsoleSender.
e0cfb347 SPIGOT-5689: Fireball.setDirection increases velocity too much
94cb030f SPIGOT-5673: swingHand API does not show to self
b331a055 SPIGOT-5680: isChunkGenerated creates empty region files
e1335932 Improve build process
a8ec1d60 Add a couple of method null checks to CraftWorld
ce66f693 Misc checkstyle fixes
8bd0e9ab SPIGOT-5669: Fix Beehive.isSedated

Spigot Changes:
2040c4c4 SPIGOT-5677, MC-114796: Fix portals generating outside world border
ab8f6b5a Rebuild patches
e7dc2f53 Rebuild patches
2020-04-27 03:34:45 -04:00
Aikar ac4f6b5022
Clean up Timings and Async Chunk Configs
Renames a bunch of timings to be more appropriate for the new environment.

Many things dealt with sync loads which wasn't correct anymore.
adjusted timings to be a little bit more accurate here.

Also cleaned up old 1.13 async chunks configs so people won't keep
thinking they can change some of those configs when they can't.
2020-04-26 02:19:43 -04:00
Aikar fcf89e852e
Improve mid tick chunk loading, Fix Oversleep, other improvements
Process loads outside of any canSleep check. Original intent was to
only apply those restrictions to generations but realized I had some
checks higher up the call chain.

Reworked the back off strategy to just run every 1 millisecond per world,
and to apply the per tick limit to generations only.

This guarantees that your chunk will load with at most around 1ms delay.

Additionally, fire midTick processing in a few more places, notably the
oversleep section so we can keep processing loads here too which has
a large up to 50ms window...

Speaking of oversleep, we had a bug in our implementation changes for
Timings that caused oversleep to not sleep the correct amount.

Because we now moved it into the NEXT tick instead of THIS tick, the
value of nextTick had already been increased to +50ms, resulting in
the risk of sleeping more than it should, but, more importantly, this
caused every task that was trying to NOT run during oversleep to actually
run during oversleep.

This is now fixed.

Another small tweak is to the /tps command, to no longer show the star when
TPS is right at 20.

Due to ineffeciencies in the sleep precision, TPS is commonly 20.02.
This causes the star to show up almost constantly, so now only show it if
we actually hit a real "catchup".

This commit also improves the changes to the CallbackExecutor, in that
it now is also recursion safe.

It was possible that the executor could run tasks out of desired order
if the executor task scheduled more executor tasks.

We solve this by ensuring new additions do not enter the currently iterated queue.

Each depth level will have its own queue.

Fixes #3220
2020-04-26 01:14:52 -04:00
Aikar cb15cfa4f8
Improve Async Login so pending connections dont get exposed
We still keep vanilla process of waiting for existing session to be removed before logging in
by storing a separate map of pending.

also fire the callback using executor incase further recursion causes any trouble
2020-04-24 05:48:51 -04:00
Aikar 509a828e06
Fix loading spawn chunks when async chunks is off
I feel sorry for those who need to do this, and now feel sorry more
since back to slow startups again.

There is keep-spawn-loaded-range in paper.yml to reduce the range to
mitigate this if you must keep async chunks off.
2020-04-22 05:47:35 -04:00
Ytnoos bf698865f7
Revert "Re-track players that dismount from other players" 2020-04-22 03:16:31 -04:00
Renamed from Spigot-Server-Patches/0391-Asynchronous-chunk-IO-and-loading.patch (Browse further)