Mouse Hit Point - RayCast

plane-line intersection

I am not aware of something already in maxscript that would let you do that, so it looks like you're gonna have to make it from scratch.

What I would do is individually take the bounding box of every object in the scene and use an imaginary plane that is coplanar with the side of the bBox who's normal vector is pointing most towards the camera.
To get the normal of the imaginary plane, do the cross product of two edges(vectors) of the bBox that are touching.
To find which normal is pointing most closely do the dot product of the look vector of the viewpor (I think its the 3rd row of the camera orientation matrix: getViewTM()), and the imaginary plane normal. Whichever one of the dot products is the smallest is most closely aligned.
At this point you will have only one plane to test per object.
Use the function bellow to see where your mouse click intersects with the imaginary planes of each object. By filtering out the collisions that are outside the bounds of the object your are testing (because the size of the planes rnt finite), and by testing which collition is nearest to the viewport camera position, you will be able to find which node is under your mouse click.

/*===========================================
planeLineIntersection
	p0, p1 = point3 verts that make up line segment
	Pp = point3 point the plane exists on
	Pn = point3 normal vector of the plane
	returns point3 intersectionPt or undefined
===========================================*/
function planeLineIntersection p0 p1 Pp Pn =
(
	local u = p1 - p0
	local w = p0 - Pp
	local D = dot(Pn)(u) --denominator of equation
	local N = -1 * dot(Pn)(w) --numerator of equation
 
	if ( D != 0 ) then
	(
		--if D == 0 then the line is parallel to the plane
		--if N == 0 then the line lies on the plane
		sI = N / D
		if ( sI > 0 or sI < 1 ) then
		(
			return ( [9999, y, z] + sI * u )
		)
	)
	return undefined
)

Sry if I made it sound more complex than it was. lol