Commit Graph

2550 Commits

Author SHA1 Message Date
Aikar ea1a909c80
Fix netty version to 50 and restore epoll native support
Accidently used the snapshot, as well as needed to exclude the
existing old 25 build of libnetty so that we load the newer build of
epoll instead.

Thanks to 56738 for the catch.
2020-05-24 13:07:53 -04:00
Aikar 10502558e9
Workaround for some hacky environments that mess up things
2 people had issues where some plugin is doing some reallly insane NMS hackery
that created invalid worlds, which caused some errors...

Really don't understand what in the world they did, but putting in a dumb guard that
shouldn't even be necessary to just not send the sound effect rather than erroring.
2020-05-23 22:27:37 -04:00
Aikar 8b2a437b9e
Fix plugins calling getChunkAtAsync asynchronously
While this method has async in it's name, it's not actually meant
to be called asynchronously.... It just means IT will load the chunk
asynchronously without blocking main.

So fix this so that if a plugin calls it async, it forces the request back to main thread.
2020-05-23 18:38:29 -04:00
Aikar c8055836e8
Update Netty to 4.1.51
Minecraft's Netty version was severely out of date. There has been
numerous security fixes, bug fixes, and likely performance fixes
since the version Minecraft uses (4.1.25).

This fixes some known issues with "Closed Channel" spam.

Fixes #3388
2020-05-23 17:58:45 -04:00
Aikar 64cfcf3e4b
Optimize sending packets to nearby locations (sounds/effects)
Instead of using the entire world or player list, use the distance
maps to only iterate players who are even seeing the chunk the packet
is originating from.

This will drastically cut down on packet sending cost for worlds with
lots of players in them.

Closes #3437
2020-05-23 17:17:14 -04:00
Aikar cf82dad399
Fix Non Full Status Chunk NBT Memory Leak
Any full status chunk that was requested for any status less than full
would hold onto their entire nbt tree and every variable in that function.

This was due to use of a lambda that persists on the Chunk object
until that chunk reaches FULL status.

With introduction of no tick, we greatly increased the number of non
full chunks so this was really starting to hurt.

We further improve it by making a copy of the nbt tag with only the memory
it needs, so that we dont have to hold a copy to the entire compound.
2020-05-23 01:40:27 -04:00
Aikar 5a12515a42
Update Log4J Configuration file to stop truncating stack traces
This should help greatly (as long as this change works...) in
understanding an exception when it doesn't get truncated with
"... and 14 more" at a vital point of the stack trace.
2020-05-22 23:52:56 -04:00
Aikar 7c001d64a5
More Improvements to Chunks
Fixed issues where urgent and prioritized chunks didn't actually
always get their priority boosted correctly....

Properly deprioritize non ticking chunks.
Limit recursion on watchdog prints to stop flooding as much
Remove neighbor priorities from watchdog to reduce information
reduce synchronization duration so that watch dog won't block main should main actually wake up
probably fixed a deadlock risk in watchdog printing also that was leading to crashes
fixed chunk holder enqueues not being processed correctly
added async catchers in some locations that should not be ran async

Fixed upstream bug where VITAL callbacks that must run on main actually could
sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately!

This build will provide massive improvements to stability as well as even faster
sync chunk load/gens now that priority is correctly set.

Fixes #3435
2020-05-22 23:52:56 -04:00
Aikar e1c451968c
Fix high memory use of non ticking chunks
The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks
as we store the NBT Tag of Tile Entities in a Chunk, but don't process
them and remove them until chunk reaches Entity Ticking status....

This caused some phantom references to persist causing high memory use
of these chunks.

So I just got rid of pooling from NBT deserialization and we'll have to
take the hit on memory allocations there because too many cascading concerns
with anyone using NBT Tag Byte Arrays.

Fixes #3431
2020-05-22 23:52:51 -04:00
Aikar ee9f0d513f
Fix another case of breaking blocks causing sync chunk loads 2020-05-22 18:57:48 -04:00
Aikar 6009ba8f44
Drop AABB limit patch until it can be tested more
Fixes #3428
2020-05-22 02:21:32 -04:00
Aikar 0e9c24e595
Fix log spam about Hanging entities bounding boxes
Nothing really "Broke" them, just a lot of log spam. This will clean that up
2020-05-22 02:04:00 -04:00
Aikar 83fadad7da
Fix conversion for deserializing raw nbt itemstacks - Fixes #3424 2020-05-22 01:08:18 -04:00
Aikar 4d38ee111f
Many fixes and improvements to chunk prioritization
I believe this brings us back to stable. A lot of complexity was
learned about juggling priorities.

We were essentially promoting more chunks to urgent than really
needed to be urgent.

So this commit adds a lot more logic to juggle neighbor priorities
and demote their priority once they meet the requirements needed of
them.

This greatly improves the performance of "urgent" chunks".

Fixes #3410
Fixes #3426
Fixes #3425
Fixes #3416
2020-05-22 01:03:42 -04:00
Aikar 281181c7c7
Use saner Entity bounding box limits
CB only protected from > 64 but there's no reason an entity should ever
be more than 2x its width or 1x height as the BB is supposed to represent
the entity size.

BB is / 2 to calculate position.
2020-05-22 00:40:56 -04:00
Aikar edd6b6a2ba
Protect the visible chunk map from plugins touching it, trim Timing Errors
Blow up if a plugin tries to mutate visibleChunks directly and prevent them
from doing so.

Also provide a safe get call if any plugins directly call get on it so
that it uses the special logic to check pending.

Also restores ABI for the visibleChunks field back to what it was too.

Additionally, remove the stack trace from Timings Stack Corruption for any
error thrown on Minecraft Timings, and tell them to get the error ABOVE this
instead, so people stop giving us useless error reports.

Also fixes a memory leak when the source map down sizes but dest map didn't,
which resulted in lingering references to old chunk holders.

Fixes #3414
2020-05-22 00:39:16 -04:00
Aikar 18c686576b
Optimize performance of object pool
synchronized arraydeque ends up still being way faster.
Kinda shocked how much that strategy was using, it wasn't really
that complicated... but oh well, this is even simpler and not
seeing blocked threads show up at all in profiling because
the lock is held for such a short amount of time.

also because most uses are on either server thread pool or chunk load pool.

Also optimize the pooling of nibbles to not register Cleaner's
for Light Engine directed usages, as we know we are properly
controlling clean up there, so we don't need to rely on GC.

This will return them to the pool manually, saving a lot of Cleaners.

Closes #3417
2020-05-20 21:45:43 -04:00
Aikar 7e1525ea2d
Many improvements to chunk prioritization and bug fixes
Fixed a few bugs, and made numerous improvements.
Fixed issue where a sync chunk load could have its ticket removed and the
priority ticket could expire...
Still not perfect there but better than before.

Also fixed few other misc issues such as watchdog cpu usage, chunk queue update
had risk of double enqueue due to it no longer being a set.

Added much more information about chunk state to watchdog prints.

I see some more room for improvement even, but this is much better than before.

Fixes #3407
Fixes #3411
Fixes #3395
Fixes #3389
2020-05-20 05:13:01 -04:00
Aikar c82b292ab0
Fix pooled buffer leak resulting in dynmap black spots - Fixes #3386
Dynmap accessed the raw bytes because it utilized NBT locally, but the
NBTTagcompound was garbage collected while the bytes were still being used.

This will return getBytes() back to being safe, and add a new PoolSafe method
that will prevent the additional allocations for general chunk loading.

Also fixed applyPatches for people with paths in their working directory
if they have mcdev sources built.
2020-05-20 00:55:30 -04:00
Aikar 614a664bd3
Implement Chunk Priority / Urgency System for Chunks
Mark chunks that are blocking main thread for world generation as urgent

Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.

Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.

This massively reduces the lag spikes from sync chunk gens.

Then we further prioritize loading order so nearby chunks have higher
priority than distant chunks, reducing the pressure a high no tick
view distance holds on you.

Chunks in front of the player have higher priority, to help with
fast traveling players keep up with their movement.

This commit also improves single core cpu scenarios in that we will
now automatically disable Async Chunks as well as Minecrafts thread
pool.

It is never recommended to use async chunks on a single CPU as context
switching will be slower than just running it all on main.

This also bumps the number of server worker threads by default too.
Mojang does not utilize the workers in an effecient manner, resulting
in them using barely any sustained CPU.

So give it more workers so more chunks can be processed concurrently

This change also improves urgent chunk loading, so players flying into
unloaded chunks will hurt a little bit less (but still hurt)

Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 04:09:37 -04:00
Mariell c8028d1c76
Fix data version check for ItemStack serialization (#3394)
The expected version should be equal to or newer than the one stored.

Although Aikar claims he did this on accident (and NOT my ligatures!), I
claim this is all a big conspiracy by followers of the Taco cult.
2020-05-18 12:24:06 -05:00
Mariell Hoversholm 6f196fe701
Add Raw Byte ItemStack Serialization
Serializes using NBT which is safer for server data migrations than bukkits format.
2020-05-17 00:52:24 -04:00
Spottedleaf df43f82838
Allow server startup for those poor people running <1G Xmx
Clamp logic for poolsize so we dont pass zero
2020-05-17 00:32:37 -04:00
Spottedleaf 3c9b65ef9d
Fix cases where no-tick < tick view distance 2020-05-17 00:24:49 -04:00
MeFisto94 72f89a076a
Workaround for Client Lag Spikes (MC-162253)
When crossing certain chunk boundaries, the client needlessly
calculates light maps for chunk neighbours. In some specific map
configurations, these calculations cause a 500ms+ freeze on the Client.

This patch basically serves as a workaround by sending light maps
to the client, so that it doesn't attempt to calculate them.
This mitigates the frametime impact to a minimum (but it's still there).
2020-05-17 00:21:42 -04:00
Mariell Hoversholm 3f941c0cce
Add option for console having all permissions
Overrides permissions set to default false (which op does not get)
2020-05-16 23:55:54 -04:00
Mariell Hoversholm d2ae4658ec
Add permission for command blocks 2020-05-16 23:48:54 -04:00
kickash32 74466412ee
Check portal restrictions when entering end gateways 2020-05-16 23:28:59 -04:00
Josh Roy fc9cf84dc6
Fix NPE when temp ip bans expire (#3373) 2020-05-16 23:26:49 -04:00
Max Lee 16bd420db5
Add missing mob goals for API (#3367) 2020-05-16 23:19:24 -04:00
Mariell Hoversholm b5c4e2f655
Ensure no-tick view is not smaller than ticking VD
Fixes #3372.
2020-05-16 23:09:06 -04:00
MiniDigger 52564b1f89
Expand Pathfinding API with more options 2020-05-16 23:02:48 -04:00
MiniDigger dde6548136
Fix usage of vanilla goals 2020-05-16 22:59:20 -04:00
kickash32 7797aebe63
Drop Leads from nether portals - Fixes #3226 2020-05-16 22:58:11 -04:00
Spottedleaf 511b6bc278
Reduce MutableInt and Vec3d allocations, use ArrayDeque 2020-05-16 22:26:45 -04:00
Aikar 84673141a0
Optimize NibbleArray to use pooled buffers
Massively reduces memory allocation of 2048 byte buffers by using
an object pool for these.

Uses lots of advanced new capabilities of the Paper codebase :)

Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration
of 6GB of heap (any more over 6GB won't give more nibble pool)

You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048

Remember this number is * 8 then * heap memory in GB

That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB

You may also control max number of pooled objects directly instead of any
dynamic calculation using -DPaper.maxNibblePoolSize=1024000

While this will use more old generation by a tad bit, allocation rate will drop
significantly, causing less young generation GC's.

This commit has gone through extensive testing for over a day and confident
it no longer has any issues with light corruption.
2020-05-16 22:24:36 -04:00
Aikar 897dd2c840
Foundational work for Future Memory usage improvements
This commit doesn't do much on its own, but adds a new Java Cleaner API
that lets us hook into Garbage Collector events to reclaim pooled objects and
return them to the pool.

Adds framework for Network Packets to know when a packet has finished dispatching
to get an idea when a packet is done sending to players.

Rewrites PooledObjects impl to properly respect max pool size and remove
almost all risk of contention.

Bumps the Paper Async Task Queue to use 2 threads, and properly shuts it down on shutdown.
2020-05-16 21:38:19 -04:00
Spottedleaf bb4002d82e
Handle CraftPlayer#setSpectatorTarget better
Use a proper teleport for teleporting to entities in different
worlds.

Validate that the target entity is valid and deny spectate
requests from frozen players.

Also, make sure the entity is spawned to the client before
sending the camera packet. If the entity isn't spawned clientside
when it receives the camera packet, then the client will not
spectate the target entity.
2020-05-14 18:05:00 -05:00
Aikar 4ae089597b
Fix collision checks on spawning hanging entities and null on async chunk loads
getCubes will now always load chunks
getChunk with gen false will now not throw error

Fixes #3368
Fixes #3364
2020-05-14 05:05:13 -04:00
Aikar c2f8d1ef98
Protect Bedrock and End Portal/Frames from being destroyed
This fixes exploits that let players destroy bedrock by Pistons, explosions
and Mushrooom/Tree generation.

These blocks are designed to not be broken except by creative players/commands.
So protect them from a multitude of methods of destroying them.

A config is provided if you rather let players use these exploits, and let
them destroy the worlds End Portals and get on top of the nether easy.
2020-05-14 03:16:31 -04:00
Aikar 827cc63263
Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
ffc8e4ca SPIGOT-5716: Clarify documentation of MultipleFacing

CraftBukkit Changes:
d07a78b1 SPIGOT-5716: Clarify documentation of MultipleFacing
46a13860 SPIGOT-5718: Block.BreakBlockNaturally does not reflect tool used
214ffea9 SPIGOT-5727: GameRule doImmediateRespawn cannot be set per-world

Spigot Changes:
2f5d615f SPIGOT-5730: Modernise inventory patch
a2bdb119 SPIGOT-5679: Add config option for end portal activation sound

Closes #3352
2020-05-12 01:27:07 -04:00
Aikar 92f680ed84
Fix Pathfinding and obscure glitchy buggy 0 tick farms
I swear the crap that stuff will abuse to make stuff happen is insane.

Hash codes apparently changing behavior of stuff based on its value, so
reverting 2d401d2dfb

Fixes #3346
Fixes #3341
2020-05-11 21:18:11 -04:00
Aikar 7a7c429260
Optimize Pathfinder - Remove Streams / Optimized collections
I utilized the IDE to convert streams to non streams code, so shouldn't
be any risk of behavior change. Only did minor optimization of the
generated code set to remove unnecessary things.

I expect us to just drop this patch on next major update and re-apply
it with the IDE again and re-apply the collections optimization.

Optimize collection by creating a list instead of a set of the key and value.

This lets us get faster foreach iteration, as well as avoids map lookups on
the values when needed.
2020-05-11 04:24:38 -04:00
Aikar fc917d1687
Optimize Hoppers - Major performance improvement
Removed streams from hoppers and also fixed a mistake in the logic.
When this patch was ported to 1.14/1.15, a line of code was put in
the wrong place which disabled a significant portion of the improvement.

Replaced usages of streams in isEmpty and itemstack checks
Replaced usage of streams in pulling loop
Replaced usage of streams in Lootable Inventory isEmpty() check
Only check for refilling Lootable Inventory when accessing first slot, not all

All of these in general were pretty significant hits, so this single commit
is going to cause tacos to magically appear in front of you every day.

🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮

Nom Nom Nom
If you hate taco's, you're not allowed to use this improvement.

Also ignore the renames, pulled a lot of PR's.
2020-05-11 03:42:32 -04:00
Omer Uddin 14ad77c692
Fix PotionEffect API Ignoring Icon bug
Fixes #3142
2020-05-11 00:44:27 -04:00
Omer Uddin eb3ce8a258
Fix EntityRaider picking up items when they shouldn't be able
Fixes #2205
2020-05-11 00:41:15 -04:00
Aikar 1ea9ada0da
Add a TELEPORT ticket when changing dimensions
This should fix any remaining issues around collision issues on
chunk edges when changing dimensions sending to Y 256
2020-05-11 00:19:40 -04:00
Aikar 8e9459eadc
Fix missing flag pass for isUrgent 2020-05-11 00:19:40 -04:00
Josh Roy 7befec4420
Potential bed api (#3339) 2020-05-11 00:19:04 -04:00
Spottedleaf 27945a6b1f
Optimize WorldBorder collision checks and air
Port part of leafs changes from Tuinity, will worry about rest later with larger changes coming
2020-05-10 22:52:20 -04:00