Move object with hinge joint, but keep it connected to hinge - unity3d

Question

I have multiple spheres in my mobile game that are attached to hinge joints. The spheres are able to swing and knock into each other, causing other spheres to swing. I am creating movement in the spheres by touching on a sphere and dragging it to a new location. Letting go is supposed to cause the sphere that I just moved to swing accordingly.
The issue is that I am able to move the spheres well outside of the space provided by the hinge. I never want the spheres to move anywhere that they wouldn't be able to swing using the hinge. I am able to move the spheres multiple units/meters away from their original position, when ideally I wouldn't be able to move them more than a few centimeters. The spheres should just stop moving if I hit a limit in the hinge.
Here's my code for the script that controls movement of the spheres:
GameObject selectedObject;
Vector3 screenPoint;
Vector3 offset;
void Update () {
if (Input.touchCount == 0)
{
return;
}
Touch touch = Input.GetTouch(0);
if (touch.phase == TouchPhase.Began) // when screen is touched...
{
RaycastHit hit;
if (Physics.Raycast(Camera.main.ScreenToWorldPoint(touch.position), Camera.main.transform.forward, out hit)) // ...cast a ray...
{
if (hit.collider.tag == "Sphere") //...and check if ray hits a sphere
{
selectedObject = hit.collider.gameObject;
screenPoint = Camera.main.WorldToScreenPoint(selectedObject.transform.position);
offset = selectedObject.transform.position - Camera.main.ScreenToWorldPoint(new Vector3(touch.position.x, touch.position.y, screenPoint.z));
}
}
}
if (touch.phase == TouchPhase.Moved)
{
Vector3 touchPoint = new Vector3(touch.position.x, touch.position.y, screenPoint.z);
Vector3 touchPosition = Camera.main.ScreenToWorldPoint(touchPoint) + offset;
selectedObject.transform.position = touchPosition;
}
}
Any help is greatly appreciated! Let me know if I need to explain more or show a video of the issue.

Answer 1

I know I am a little late but you can use AddForce() and AddTorque() methods on the object you want to move but that works only if you have RigidBody on the object.